Lingolol

Harald SackThe <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> lecture is over. Today our students will write the final exam. FIngers crossed. Meanwhile - as our favorite coffee place is on vacation - Cappuccino and Zupfkuchen (highly recommended) together with the <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> TA team at Intro Café :) <a href="https://sigmoid.social/tags/coffeechallenge" class="mention hashtag" rel="nofollow noopener" target="_blank">#coffeechallenge</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://sigmoid.social/tags/academiclife" class="mention hashtag" rel="nofollow noopener" target="_blank">#academiclife</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://sigmoid.social/@GenAsefa" class="u-url mention" rel="nofollow noopener" target="_blank">@GenAsefa</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://social.kit.edu/@KIT_Karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@KIT_Karlsruhe</a>

Harald SackIn our last <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> lecture last week, we were discussing what makes a node "important" in a knowledge graph. A simple heuristics can be borrowed from graph theory or communication theory: Degree CentralityInterestingly, in Wikidata In-degree centrality states Jane Austen to be to most "important" female author, while Out-degree centrality claims J.K. Rowling as being more "important" ;-) <a href="https://sigmoid.social/tags/knowledgegraphs" class="mention hashtag" rel="nofollow noopener" target="_blank">#knowledgegraphs</a> <a href="https://sigmoid.social/tags/semanticweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#semanticweb</a> <a href="https://sigmoid.social/tags/graphtheory" class="mention hashtag" rel="nofollow noopener" target="_blank">#graphtheory</a> <a href="https://sigmoid.social/tags/feminism" class="mention hashtag" rel="nofollow noopener" target="_blank">#feminism</a> <a href="https://sigmoid.social/tags/eyeofthebeholder" class="mention hashtag" rel="nofollow noopener" target="_blank">#eyeofthebeholder</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a>

Harald SackOne of our final topics in the <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> lecture were Knowledge Graph Embeddings. How to vectorise KG structures while preserving their inherent semantics? <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://sigmoid.social/tags/KGE" class="mention hashtag" rel="nofollow noopener" target="_blank">#KGE</a> <a href="https://sigmoid.social/tags/knowledgegraphs" class="mention hashtag" rel="nofollow noopener" target="_blank">#knowledgegraphs</a> <a href="https://sigmoid.social/tags/embeddings" class="mention hashtag" rel="nofollow noopener" target="_blank">#embeddings</a> <a href="https://sigmoid.social/tags/semanticweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#semanticweb</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a>

Harald SackThis week's ISE 2025 lecture was focussed on artificial neural networks. In particular, we were discussing how to get rid of manual feature engineering and doing representation learning from raw data with convolutional neural networks.<a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://sigmoid.social/tags/ArtificialNeuralNetworks" class="mention hashtag" rel="nofollow noopener" target="_blank">#ArtificialNeuralNetworks</a> <a href="https://sigmoid.social/tags/cnn" class="mention hashtag" rel="nofollow noopener" target="_blank">#cnn</a> <a href="https://sigmoid.social/tags/deeplearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#deeplearning</a> <a href="https://sigmoid.social/tags/machinelearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#machinelearning</a> <a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://mastodon.social/@sarahjamielewis" class="u-url mention" rel="nofollow noopener" target="_blank">@sarahjamielewis</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a>

Harald SackIn the ISE2025 lecture today, our students learned about unsupervised learning on the example of k-Means clustering. One nice hands-on example is image colour reduction based on k-means clustering, as demonstrated in a colab notebook (based on the Python DataScience Handbook by Vanderplus)colab notebook: <a href="https://colab.research.google.com/drive/1lhdq2pynuwJKoXbspydECuWcPRw3-xxn?usp=sharing" rel="nofollow noopener" translate="no" target="_blank">https://colab.research.google.com/drive/1lhdq2pynuwJKoXbspydECuWcPRw3-xxn?usp=sharing</a> Python DataScience Handbook: <a href="https://archive.org/details/python-data-science-handbook.pdf/mode/2up" rel="nofollow noopener" translate="no" target="_blank">https://archive.org/details/python-data-science-handbook.pdf/mode/2up</a><a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://sigmoid.social/tags/datascience" class="mention hashtag" rel="nofollow noopener" target="_blank">#datascience</a> <a href="https://sigmoid.social/tags/machinelearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#machinelearning</a> <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://social.kit.edu/@KIT_Karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@KIT_Karlsruhe</a>

Harald SackTomorrow, we will dive deeper into ontologies with OWL, the Web Ontology Language. However, I'm doing OWL-lectures now for almost 20 years - and OWL as well as the lecture haven't changed much. So, I'm afraid I'm going to surprise/dissapoint the students tomorrow, when I will switch off the presentation and start improvising a random OWL ontology with them on the blackboard ;-) <a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/tags/OWL" class="mention hashtag" rel="nofollow noopener" target="_blank">#OWL</a> <a href="https://sigmoid.social/tags/semanticweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#semanticweb</a> <a href="https://sigmoid.social/tags/semweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#semweb</a> <a href="https://sigmoid.social/tags/RDF" class="mention hashtag" rel="nofollow noopener" target="_blank">#RDF</a> <a href="https://sigmoid.social/tags/knowledgegraphs" class="mention hashtag" rel="nofollow noopener" target="_blank">#knowledgegraphs</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a>

Sarven Capadisli<a href="https://sigmoid.social/@lysander07" class="u-url mention" rel="nofollow noopener" target="_blank">@lysander07</a> What I like about this view is that info is not necessarily complete. Typical real-world data. So, some queries will not show everything there is to know about a topic while a lot of info may be for other things. Though it can be extended - "pay as you go".Some students may also be interested in expressing their research output as a <a href="https://w3c.social/tags/KnowledgeGraph" class="mention hashtag" rel="nofollow noopener" target="_blank">#KnowledgeGraph</a> on its own right - playing an important role in <a href="https://w3c.social/tags/ScholarlyCommunication" class="mention hashtag" rel="nofollow noopener" target="_blank">#ScholarlyCommunication</a>eg view: <a href="https://dokie.li/?graph=https://csarven.ca/linked-research-decentralised-web" rel="nofollow noopener" translate="no" target="_blank">https://dokie.li/?graph=https://csarven.ca/linked-research-decentralised-web</a>seeAlso alt text.<a href="https://w3c.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a>

Harald SackIn today's ISE 2025 lecture,, we will introduce SPARQL as a query language for knowledge graphs. Again, I'm trying out 'Dystopian Novels' as example knowledge graph playground. Let's see, if the students might know any of them. Wtat do you think? ;-) <a href="https://sigmoid.social/tags/dystopia" class="mention hashtag" rel="nofollow noopener" target="_blank">#dystopia</a> <a href="https://sigmoid.social/tags/literature" class="mention hashtag" rel="nofollow noopener" target="_blank">#literature</a> <a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/tags/semanticweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#semanticweb</a> <a href="https://sigmoid.social/tags/semweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#semweb</a> <a href="https://sigmoid.social/tags/knowledgegraphs" class="mention hashtag" rel="nofollow noopener" target="_blank">#knowledgegraphs</a> <a href="https://sigmoid.social/tags/sparql" class="mention hashtag" rel="nofollow noopener" target="_blank">#sparql</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a>

Harald SackBack in the lecture hall again after two exciting weeks of <a href="https://sigmoid.social/tags/ESWC2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ESWC2025</a> and <a href="https://sigmoid.social/tags/ISWS2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISWS2025</a>. This morning, we introduced our students to RDF, RDFS, RDF Inferencing, and RDF Reification.<a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/tags/semanticweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#semanticweb</a> <a href="https://sigmoid.social/tags/semweb" class="mention hashtag" rel="nofollow noopener" target="_blank">#semweb</a> <a href="https://sigmoid.social/tags/knowledgegraphs" class="mention hashtag" rel="nofollow noopener" target="_blank">#knowledgegraphs</a> <a href="https://sigmoid.social/tags/rdf" class="mention hashtag" rel="nofollow noopener" target="_blank">#rdf</a> <a href="https://sigmoid.social/tags/reasoning" class="mention hashtag" rel="nofollow noopener" target="_blank">#reasoning</a> <a href="https://sigmoid.social/tags/reification" class="mention hashtag" rel="nofollow noopener" target="_blank">#reification</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://mastodon.social/@KIT_Karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@KIT_Karlsruhe</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a>

Harald SackLast week, we continued our <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> lecture on distributional semantics with the introduction of neural language models (NLMs) and compared them to traditional statistical n-gram models. Benefits of NLMs: - Capturing Long-Range Dependencies - Computational and Statistical Tractability - Improved Generalisation - Higher Accuracy<a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://sigmoid.social/tags/llms" class="mention hashtag" rel="nofollow noopener" target="_blank">#llms</a> <a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="nofollow noopener" target="_blank">#nlp</a> <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a>

Harald SackIn the <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> lecture today we were introducing our students to the concept of distributional semantics as the foundation of modern large language models. Historically, Wittgenstein was one of the important figures in the Philosophy of Language stating thet "The meaning of a word is its use in the language."<a href="https://static1.squarespace.com/static/54889e73e4b0a2c1f9891289/t/564b61a4e4b04eca59c4d232/1447780772744/Ludwig.Wittgenstein.-.Philosophical.Investigations.pdf" rel="nofollow noopener" translate="no" target="_blank">https://static1.squarespace.com/static/54889e73e4b0a2c1f9891289/t/564b61a4e4b04eca59c4d232/1447780772744/Ludwig.Wittgenstein.-.Philosophical.Investigations.pdf</a><a href="https://sigmoid.social/tags/philosophy" class="mention hashtag" rel="nofollow noopener" target="_blank">#philosophy</a> <a href="https://sigmoid.social/tags/wittgenstein" class="mention hashtag" rel="nofollow noopener" target="_blank">#wittgenstein</a> <a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="nofollow noopener" target="_blank">#nlp</a> <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://sigmoid.social/tags/llm" class="mention hashtag" rel="nofollow noopener" target="_blank">#llm</a> <a href="https://sigmoid.social/tags/languagemodel" class="mention hashtag" rel="nofollow noopener" target="_blank">#languagemodel</a> <a href="https://sigmoid.social/tags/language" class="mention hashtag" rel="nofollow noopener" target="_blank">#language</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/tags/AIart" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIart</a>

Harald SackGenerating Shakespeare-like text with an n-gram language model is straight forward and quite simple. But, don't expect to much of it. It will not be able to recreate a lost Shakespear play for you ;-) It's merely a parrot, making up well sounding sentences out of fragments of original Shakespeare texts...<a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="nofollow noopener" target="_blank">#nlp</a> <a href="https://sigmoid.social/tags/llm" class="mention hashtag" rel="nofollow noopener" target="_blank">#llm</a> <a href="https://sigmoid.social/tags/languagemodel" class="mention hashtag" rel="nofollow noopener" target="_blank">#languagemodel</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/tags/shakespeare" class="mention hashtag" rel="nofollow noopener" target="_blank">#shakespeare</a> <a href="https://sigmoid.social/tags/generativeAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#generativeAI</a> <a href="https://sigmoid.social/tags/statistics" class="mention hashtag" rel="nofollow noopener" target="_blank">#statistics</a>

Harald SackIn our <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> lecture last Wednesday, we learned how in n-gram language models via Markov assumption and maximum likelihood estimation we can predict the probability of the occurrence of a word given a specific context (i.e. n words previous in the sequence of words).<a href="https://sigmoid.social/tags/NLP" class="mention hashtag" rel="nofollow noopener" target="_blank">#NLP</a> <a href="https://sigmoid.social/tags/languagemodels" class="mention hashtag" rel="nofollow noopener" target="_blank">#languagemodels</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> @tabea <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://mastodon.social/@KIT_Karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@KIT_Karlsruhe</a>

Harald SackThis week, we were discussing the central question Can we "predict" a word? as the basis for statistical language models in our <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> lecture. Of course, I wasx trying Shakespeare quotes to motivate the (international) students to complement the quotes with "predicted" missing words ;-)"All the world's a stage, and all the men and women merely...."<a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="nofollow noopener" target="_blank">#nlp</a> <a href="https://sigmoid.social/tags/llms" class="mention hashtag" rel="nofollow noopener" target="_blank">#llms</a> <a href="https://sigmoid.social/tags/languagemodel" class="mention hashtag" rel="nofollow noopener" target="_blank">#languagemodel</a> <a href="https://sigmoid.social/tags/Shakespeare" class="mention hashtag" rel="nofollow noopener" target="_blank">#Shakespeare</a> <a href="https://sigmoid.social/tags/AIart" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIart</a> lecture <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/tags/brushUpYourShakespeare" class="mention hashtag" rel="nofollow noopener" target="_blank">#brushUpYourShakespeare</a>

Harald SackLast week, our students learned how to conduct a proper evaluation for an NLP experiment. To this end, we introduced a small textcorpus with sentences about Joseph Fourier, who counts as one of the discoverers of the greenhouse effect, responsible for global warming.<a href="https://github.com/ISE-FIZKarlsruhe/ISE-teaching/blob/b72690d38911b37748082256b61f96cf86171ace/materials/dataset/fouriercorpus.txt" rel="nofollow noopener" translate="no" target="_blank">https://github.com/ISE-FIZKarlsruhe/ISE-teaching/blob/b72690d38911b37748082256b61f96cf86171ace/materials/dataset/fouriercorpus.txt</a><a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="nofollow noopener" target="_blank">#nlp</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://sigmoid.social/tags/climatechange" class="mention hashtag" rel="nofollow noopener" target="_blank">#climatechange</a> <a href="https://sigmoid.social/tags/globalwarming" class="mention hashtag" rel="nofollow noopener" target="_blank">#globalwarming</a> <a href="https://sigmoid.social/tags/historyofscience" class="mention hashtag" rel="nofollow noopener" target="_blank">#historyofscience</a> <a href="https://sigmoid.social/tags/climate" class="mention hashtag" rel="nofollow noopener" target="_blank">#climate</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a>

Harald SackLast leg on our brief history of NLP (so far) is the advent of large language models with GPT-3 in 2020 and the introduction of learning from the prompt (aka few-shot learning).T. B. Brown et al. (2020). Language models are few-shot learners. NIPS'20<a href="https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf" rel="nofollow noopener" translate="no" target="_blank">https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf</a><a href="https://sigmoid.social/tags/llms" class="mention hashtag" rel="nofollow noopener" target="_blank">#llms</a> <a href="https://sigmoid.social/tags/gpt" class="mention hashtag" rel="nofollow noopener" target="_blank">#gpt</a> <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="nofollow noopener" target="_blank">#nlp</a> <a href="https://sigmoid.social/tags/historyofscience" class="mention hashtag" rel="nofollow noopener" target="_blank">#historyofscience</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a>

Harald SackNext stop in our NLP timeline is 2013, the introduction of low dimensional dense word vectors - so-called "word embeddings" - based on distributed semantics, as e.g. word2vec by Mikolov et al. from Google, which enabled representation learning on text.T. Mikolov et al. (2013). Efficient Estimation of Word Representations in Vector Space. <a href="https://arxiv.org/abs/1301.3781" rel="nofollow noopener" translate="no" target="_blank">https://arxiv.org/abs/1301.3781</a><a href="https://sigmoid.social/tags/NLP" class="mention hashtag" rel="nofollow noopener" target="_blank">#NLP</a> <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://sigmoid.social/tags/wordembeddings" class="mention hashtag" rel="nofollow noopener" target="_blank">#wordembeddings</a> <a href="https://sigmoid.social/tags/word2vec" class="mention hashtag" rel="nofollow noopener" target="_blank">#word2vec</a> <a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/tags/historyofscience" class="mention hashtag" rel="nofollow noopener" target="_blank">#historyofscience</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a>

Harald SackBuilding on the 90s, statistical n-gram language models, trained on vast text collections, became the backbone of NLP research. They fueled advancements in nearly all NLP techniques of the era, laying the groundwork for today's AI. F. Jelinek (1997), Statistical Methods for Speech Recognition, MIT Press, Cambridge, MA<a href="https://sigmoid.social/tags/NLP" class="mention hashtag" rel="nofollow noopener" target="_blank">#NLP</a> <a href="https://sigmoid.social/tags/LanguageModels" class="mention hashtag" rel="nofollow noopener" target="_blank">#LanguageModels</a> <a href="https://sigmoid.social/tags/HistoryOfAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#HistoryOfAI</a> <a href="https://sigmoid.social/tags/TextProcessing" class="mention hashtag" rel="nofollow noopener" target="_blank">#TextProcessing</a> <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://sigmoid.social/tags/historyofscience" class="mention hashtag" rel="nofollow noopener" target="_blank">#historyofscience</a> <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a>

Harald SackNext stop on our NLP timeline (as part of the <a href="https://sigmoid.social/tags/ISE2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ISE2025</a> lecture) was Terry Winograd's SHRDLU, an early natural language understanding system developed in 1968-70 that could manipulate blocks in a virtual world. Winograd, T. Procedures as a Representation for Data in a Computer Program for Understanding Natural Language. MIT AI Technical Report 235. <a href="http://dspace.mit.edu/bitstream/handle/1721.1/7095/AITR-235.pdf" rel="nofollow noopener" translate="no" target="_blank">http://dspace.mit.edu/bitstream/handle/1721.1/7095/AITR-235.pdf</a><a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="nofollow noopener" target="_blank">#nlp</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://sigmoid.social/tags/historyofscience" class="mention hashtag" rel="nofollow noopener" target="_blank">#historyofscience</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a>

Harald SackWith the advent of ELIZA, Joseph Weizenbaum's first psychotherapist chatbot, NLP took another major step with pattern-based substitution algorithms based on simple regular expressions.Weizenbaum, Joseph (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Com. of the ACM. 9: 36–45.<a href="https://dl.acm.org/doi/pdf/10.1145/365153.365168" rel="nofollow noopener" translate="no" target="_blank">https://dl.acm.org/doi/pdf/10.1145/365153.365168</a><a href="https://sigmoid.social/tags/nlp" class="mention hashtag" rel="nofollow noopener" target="_blank">#nlp</a> <a href="https://sigmoid.social/tags/lecture" class="mention hashtag" rel="nofollow noopener" target="_blank">#lecture</a> <a href="https://sigmoid.social/tags/chatbot" class="mention hashtag" rel="nofollow noopener" target="_blank">#chatbot</a> <a href="https://sigmoid.social/tags/llm" class="mention hashtag" rel="nofollow noopener" target="_blank">#llm</a> <a href="https://sigmoid.social/tags/ise2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#ise2025</a> <a href="https://sigmoid.social/tags/historyofScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#historyofScience</a> <a href="https://sigmoid.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://sigmoid.social/@fizise" class="u-url mention" rel="nofollow noopener" target="_blank">@fizise</a> <a href="https://wisskomm.social/@fiz_karlsruhe" class="u-url mention" rel="nofollow noopener" target="_blank">@fiz_karlsruhe</a> <a href="https://fedihum.org/@tabea" class="u-url mention" rel="nofollow noopener" target="_blank">@tabea</a> <a href="https://sigmoid.social/@enorouzi" class="u-url mention" rel="nofollow noopener" target="_blank">@enorouzi</a> <a href="https://fedihum.org/@sourisnumerique" class="u-url mention" rel="nofollow noopener" target="_blank">@sourisnumerique</a>

Recent searches

Search options

Administered by:

Server stats:

#ise2025