skip to main content
Language:
Search Limited to: Search Limited to: Resource type Show Results with: Show Results with: Search type Index

Word2Vec: Optimal hyperparameters and their impact on natural language processing downstream tasks

Open computer science, 2022-03, Vol.12 (1), p.134-141 [Peer Reviewed Journal]

ISSN: 2299-1093 ;EISSN: 2299-1093 ;DOI: 10.1515/comp-2022-0236

Full text available

Citations Cited by
  • Title:
    Word2Vec: Optimal hyperparameters and their impact on natural language processing downstream tasks
  • Author: Adewumi, Tosin ; Liwicki, Foteini ; Liwicki, Marcus
  • Subjects: embeddings ; hyperparameters ; Machine Learning ; Maskininlärning ; named entity recognition ; sentiment analysis ; Word2Vec
  • Is Part Of: Open computer science, 2022-03, Vol.12 (1), p.134-141
  • Description: Word2Vec is a prominent model for natural language processing tasks. Similar inspiration is found in distributed embeddings (word-vectors) in recent state-of-the-art deep neural networks. However, wrong combination of hyperparameters can produce embeddings with poor quality. The objective of this work is to empirically show that Word2Vec optimal combination of hyper-parameters exists and evaluate various combinations. We compare them with the publicly released, original Word2Vec embedding. Both intrinsic and extrinsic (downstream) evaluations are carried out, including named entity recognition and sentiment analysis. Our main contributions include showing that the best model is usually task-specific, high analogy scores do not necessarily correlate positively with 1 scores, and performance is not dependent on data size alone. If ethical considerations to save time, energy, and the environment are made, then relatively smaller corpora may do just as well or even better in some cases. Increasing the dimension size of embeddings after a point leads to poor quality or performance. In addition, using a relatively small corpus, we obtain better WordSim scores, corresponding Spearman correlation, and better downstream performances (with significance tests) compared to the original model, which is trained on a 100 billion-word corpus.
  • Publisher: De Gruyter
  • Language: English
  • Identifier: ISSN: 2299-1093
    EISSN: 2299-1093
    DOI: 10.1515/comp-2022-0236
  • Source: SWEPUB Freely available online
    Walter De Gruyter: Open Access Journals
    DOAJ Directory of Open Access Journals
Back to results list
INSPIRE LIBRARY - TON DUC THANG UNIVERSITY (84-028) 37 755 057 Feedback
19 Nguyen Huu Tho St. Dist.7, HCM thuvien@tdtu.edu.vn

Copyright © 2017 INSPIRE LIBRARY - TON DUC THANG UNIVERSITY

Searching Remote Databases, Please Wait