We have three surfaces on Pinterest: home feed, related Pins, and search.

Eventually replace all open-sourced embeddings to save maintenance cost.

Learn a shared word embedding for all downstream tasks instead of an end-to-end embedding coupled with a particular application.

Do word-level instead of character-level embedding to make sure the learned embedding dictionary can be loaded into memory when necessary.

Use supervised information instead of unsupervised co-occurrence style word embedding like word2vec.

These gaps motivate us to do text embedding at a practical system level.

Although pre-trained word embeddings provide a good baseline, we do observe a clear gap between industrial applications and academic research, including storage cost, memory cost, availability of supervised information, throughput and latency. In Pinterest, textual data is more often about concrete annotation terms and short phrases rather than long sentences and paragraphs, which makes word embedding a more fundamental component than complex neural network architectures for specific tasks. It is known that high-quality word representation usually contributes most to text classification or other general tasks. Researchers not only proposed principled algorithms and open-source code, but they also released pre-trained embedding models with public data corpora like Wikipedia, Twitter, or Google news. Word embeddings have been actively studied and developed by researchers and practitioners in the machine learning community since the neural network language model was proposed. Jinfeng Zhuang | Software Engineer, Content Knowledge

PinText: A Multitask Text Embedding System in Pinterest

Pinterest google sign in