List of academic search engines that use Large Language models for generative answers (for the latest version - see this page)

One of the earliest themes of this blog was to track tools that not only helped with discovery but also with delivery, helping users to gain access to full-text via institution subscriptions (and of course via Open Access) even if they did not start off the library homepage

I've been a subscriber of OpenAI's ChatGPT plus for a while though I have been struggling to justify it to myself for a while until recently.

I was asked to give a talk at CILIP Conference 2023 for the Data & AI panel. I was given 10 minutes to send a recording which I eventually delivered.

As expected more academic search engines are starting to adopt the near-human Natural language understanding and natural language generation capabilities of Large Language Models.

In recent years, Elsevier has provided relatively good access to their content via APIs, whether it be via the Scopus API for metadata including bibliometrics or even full-text via Sciencedirect API as long as you are an institutional customer, typically at no additional fee.

Edit: Since writing this I have watched the Video - State of GPT | BRK216HFS by Andrej Karpathy(Open AI). In 42 minutes, it not only covers the basic high-level details on how auto-regressive decoder only models work (e.g.; GPT models) but has the clearest explanation on why prompt engineering works.

ChatGPT by OpenAI is the most famous and most used auto regressive decoder Transformer based Large Language Model.

The story so far. ChatGPT, or other large language models, is not ideal for information retrieval alone because it often hallucinates or fabricates information, including references.

Warning: I am not a data or information scientist; this is a new area I am trying to learn about The rise of large language models, such as GPT-3, GPT-4, ChatGPT, and BERT, has dramatically improved information retrieval in recent years.

Warning: I wrote this to explain to myself high-level technical details in deep learning, it is likely to include inaccuracies.