About LLMs
Useful Links
To learn more about LLMs in the context of Glycomics Workbench, check this site periodically. GenAI is based on Deep Learning. To learn about it, check here.
The NIH Common Fund's Bridge to Artificial Intelligence (Bridge2AI) aims to advance biomedical research by paving the way for widespread adoption of artificial intelligence (AI) capable of addressing intricate biomedical challenges that go beyond human intuition. Read more. Also, see NAIRR Pilot Project.
Relevant Societies
-
-
Send an email to 'workofthefuture@mit.edu', if you are interested to join the Working Group on Generative AI at MIT.
Calendar of Events
About Large Language Models
A large language model (LLM) is a statistical language model, trained on a massive amount of data, that can be used to generate and translate text and other content, and perform other natural language processing (NLP) tasks. Thus, a large language model (LLM) is used to train GenAI, a type of machine learning, to understand and generate text just like a human. There are multiple models (LLMs) for GenAI as described below. LLMs, however, differ from other machine learning models.
Since the groundbreaking paper “Attention Is All You Need” by Vaswani, et. al. (2017) of Google's BRAIN team, a lot of interest has been generated in this field of machine learning. Their seminal work introduced the Transformer model (an improved version of neural network), which is based solely on attention mechanisms. A LLM is a type of transformer model that looks for patterns in sequential data sets (like words in a sentence) to establish context. The algorithm outputs an appropriate, human-like response when presented with a text prompt. Examples of LLMs are ChatGPT from OpenAI, Copilot from Microsoft, or Gemini from Google. There are many more (see, the list). Among those, Llama from Meta, which is open-source, has generated interest among the developers.
This technology is developed by a team of dedicated and talented computer scientists. Apparently, this sort of research started since the 1960s when Joseph Weizenbaum of MIT created ELIZA, the world's first chatbot. Early chatbot technology can respond to simple questions with scripted answers but lacks true 'intelligence'. However, machine learning (ML) advancements in the 2010s have led to more advanced chatbots, like Siri and Alexa, which can understand complex language, learn from past interactions and generate creative content. In the late 2010s, advancements in machine learning (ML) paved the way for GenAI chatbots, such as Jasper AI, ChatGPT and Bard. These ML advancements let developers train chatbots on massive data sets, which help them understand natural language better than previous agents. Additionally, this advanced technology can generate creative texts, such as poems, song lyrics, short stories and essays. Transformer models are made up of layers that can be stacked to create increasingly complex algorithms. LLMs, in particular, rely on two key features of transformer models: positional encoding and self-attention. Positional encoding allows the model to analyze text non-sequentially to identify patterns. Self-attention assigns a weight to each input that determines its importance compared to the rest of the data. That way, the model can pick out the most important parts in large amounts of text. Through extensive unsupervised learning, LLMs can reliably predict the next word in a sentence based on the rules of grammar in human languages. Grammatical rules are not preprogrammed into a large language model; the algorithm infers grammar as it reads text.
Processing of massive data sets needs massive computing power and distributed computing services, such as, cloud computing. Based on web services, such GenAI programs, fetch data mostly from publicly available large data sources (databases, websites, etc.). This has also created an oppotunity to create more stronger processors. NVIDIA, for example, has recently announced its RTX GPUs, which offer up to 1,400 Tensor TFLOPS for AI inferencing. When optimized, generative AI models can run up to 5x faster than on present competing devices.
Presently, GenAI can perform the following tasks: Content generation: Writing stories, poems, scripts, and marketing materials Summarization: Compiling notes or transcripts for meetings Translation: Interpreting between human languages as well as computer languages Classification: Generating lists and analyzing text for tone (positive, negative, or neutral)
There are some pros and cons, however, as explained in an article.
Because LLM algorithms learn language by identifying relationships between words, they are not limited to one human language. Likewise, LLMs don't need to be trained for any specific skill. Thus, LLMs have a lot of flexibility in understanding the nuances of human language. On the other hand, LLMs require a lot of test data before they can be useful. For example, GPT-4 developed by OpenAI was trained using books, articles, and other text available on the internet before it was released to the public. Even though the learning process is unsupervised, human expertise is still needed to develop and maintain LLMs (see this article). The large amount of data necessary to train LLMs also presents a challenge, especially when dealing with sensitive information like healthcare, which is regulated by HIPAA.
This innovative technology is going to transform research and education.