Guiding text-to-text privatization by syntax
Arnold S, Yesilbas D, Weinzierl S (2023)
Publication Language: English
Publication Type: Conference contribution, Conference Contribution
Publication year: 2023
Pages Range: 151-162
Conference Proceedings Title: Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing
Event location: Toronto
URI: https://aclanthology.org/2023.trustnlp-1.14/
DOI: 10.18653/v1/2023.trustnlp-1.14
Abstract
Metric Differential Privacy is a generalization of differential privacy tailored to address the unique challenges of text-to-text privatization. By adding noise to the representation of words in the geometric space of embeddings, words are replaced with words located in the proximity of the noisy representation. Since embeddings are trained based on word co-occurrences, this mechanism ensures that substitutions stem from a common semantic context. Without considering the grammatical category of words, however, this mechanism cannot guarantee that substitutions play similar syntactic roles. We analyze the capability of text-to-text privatization to preserve the grammatical category of words after substitution and find that surrogate texts consist almost exclusively of nouns. Lacking the capability to produce surrogate texts that correlate with the structure of the sensitive texts, we encompass our analysis by transforming the privatization step into a candidate selection problem in which substitutions are directed to words with matching grammatical properties. We demonstrate a substantial improvement in the performance of downstream tasks by up to 4.66% while retaining comparative privacy guarantees.
Authors with CRIS profile
Stefan Arnold
Lehrstuhl für Wirtschaftsinformatik, insbesondere IT-Management
Dilara Yesilbas
Lehrstuhl für Wirtschaftsinformatik, insbesondere IT-Management
Sven Weinzierl
Lehrstuhl für Digital Industrial Service Systems
Related research project(s)
How to cite
APA:
Arnold, S., Yesilbas, D., & Weinzierl, S. (2023). Guiding text-to-text privatization by syntax. In Association for Computational Linguistics (Eds.), Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (pp. 151-162). Toronto, CA.
MLA:
Arnold, Stefan, Dilara Yesilbas, and Sven Weinzierl. "Guiding text-to-text privatization by syntax." Proceedings of the Annual Meeting of the Association for Computational Linguistics, Toronto Ed. Association for Computational Linguistics, 2023. 151-162.
BibTeX: Download