Fork me on GitHub

Adversarial representation learning for private speech generation

Schematic diagram of our model: PCMelGAN.

As more and more data is collected in various settings across organizations, companies, and countries, there has been an increase in the demand of user privacy. Developing privacy preserving methods for data analytics is thus an important area of research. In this work we present a model based on generative adversarial networks (GANs) that learns to obfuscate specific sensitive attributes in speech data. We train a model that learns to hide sensitive information in the data, while preserving the meaning in the utterance. The model is trained in two steps: first to filter sensitive information in the spectrogram domain, and then to generate new and private information independent of the filtered one. The model is based on a U-Net CNN that takes mel-spectrograms as input. A MelGAN is used to invert the spectrograms back to raw audio waveforms. We show that it is possible to hide sensitive information such as gender by generating new data, trained adversarially to maintain utility and realism.

David Ericsson, Adam Östberg, Edvin Listo Zec, John Martinsson, Olof Mogren

The Workshop on Self-supervision in Audio and Speech at the 37th International Conference on Machine Learning
PDF Fulltext

Olof Mogren, PhD, RISE Research institutes of Sweden. Follow me on Mastodon.