Guiding text-to-text privatization by syntax

Arnold S, Yesilbas D, Weinzierl S (2023)


Publication Language: English

Publication Type: Conference contribution, Conference Contribution

Publication year: 2023

Pages Range: 151-162

Conference Proceedings Title: Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing

Event location: Toronto CA

URI: https://aclanthology.org/2023.trustnlp-1.14/

DOI: 10.18653/v1/2023.trustnlp-1.14

Abstract

Metric Differential Privacy is a generalization of differential privacy tailored to address the unique challenges of text-to-text privatization. By adding noise to the representation of words in the geometric space of embeddings, words are replaced with words located in the proximity of the noisy representation. Since embeddings are trained based on word co-occurrences, this mechanism ensures that substitutions stem from a common semantic context. Without considering the grammatical category of words, however, this mechanism cannot guarantee that substitutions play similar syntactic roles. We analyze the capability of text-to-text privatization to preserve the grammatical category of words after substitution and find that surrogate texts consist almost exclusively of nouns. Lacking the capability to produce surrogate texts that correlate with the structure of the sensitive texts, we encompass our analysis by transforming the privatization step into a candidate selection problem in which substitutions are directed to words with matching grammatical properties. We demonstrate a substantial improvement in the performance of downstream tasks by up to 4.66% while retaining comparative privacy guarantees.

Authors with CRIS profile

Related research project(s)

How to cite

APA:

Arnold, S., Yesilbas, D., & Weinzierl, S. (2023). Guiding text-to-text privatization by syntax. In Association for Computational Linguistics (Eds.), Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (pp. 151-162). Toronto, CA.

MLA:

Arnold, Stefan, Dilara Yesilbas, and Sven Weinzierl. "Guiding text-to-text privatization by syntax." Proceedings of the Annual Meeting of the Association for Computational Linguistics, Toronto Ed. Association for Computational Linguistics, 2023. 151-162.

BibTeX: Download