JabRef
v6
v6
  • JabRef Bibliography Management
  • Installation
  • Getting started
  • Collect
    • Add entry manually
    • Add entry using an ID
    • Add entry using reference text
    • Searching externally using Online Services
    • Add entry using PDFs
    • Add PDFs to an entry
    • Browser Extension
    • Import
      • Custom import filters
      • Import inspection window
  • Organize
    • Edit an entry
    • Groups
    • Keywords
    • Mark and grade
    • Comment on an entry
    • Searching within the library
    • Complete information using online databases
    • Manage associated files
    • Manage field names and their content
    • Best practices
    • Check consistency
    • Cleanup entries
    • Check integrity
    • Find duplicates
    • Merge entries
    • Save actions
  • Cite
    • BibTeX and biblatex
    • Pushing to external editor application
    • Export to Microsoft Word
    • OpenOffice/LibreOffice integration
  • Share
    • Sharing a Bib(la)TeX Library
    • Shared SQL Database
      • Migration of pre-3.6 SQL databases into a shared SQL database
    • Export
      • Custom export filters
    • Send as email
  • AI functionality
    • AI providers and API keys
    • AI preferences
    • AI troubleshooting
    • Running a local LLM model
  • Configuration
    • Customize the citation key generator
    • Customize entry types
    • Customize general fields
    • Customize key bindings
    • Library properties
    • Entry preview setup
    • Manage external file types
    • Manage protected terms
    • The string editor
  • Advanced information
    • Main Window
    • Entry Editor
      • Links to other entries
      • The Bibtex / Biblatex source tab
      • The 'owner' field
      • Time stamp field
      • LaTeX Citations Tab
    • About BibTeX and its fields
    • Strings
    • Field content selector
    • URL and DOI links in JabRef
    • Command line use and options
    • Automatic Backup (.sav and .bak) and Autosave
    • XMP metadata support in JabRef
    • Remote operation
    • Custom themes
    • Journal abbreviations
    • New subdatabase based on AUX file
    • How to expand first names of an entry
    • Debugging your library file
    • Resources
    • License
    • Knowledge
      • MS Office Bibliography XML format
      • Comparison of the Medline (txt), Medline (xml), and RIS format
      • EndNote Export Filter
  • Frequently Asked Questions
    • Linux
    • Mac OS X
    • Windows
  • JabKit
  • Discussion Forum
  • Contribute to JabRef
    • How to Improve the Help Page
    • How to translate the JabRef User Interface
Powered by GitBook
On this page
  • High-level explanation
  • Step-by-step guide for ollama
  • Step-by-step guide for GPT4All

Was this helpful?

Edit on GitHub
Export as PDF
  1. AI functionality

Running a local LLM model

PreviousAI troubleshootingNextConfiguration

Last updated 3 months ago

Was this helpful?

Notice:

  1. This tutorial is intended for expert users

  2. (Local) LLMs require a lot of computational power

  3. Smaller models (in terms of parameter size) typically respond qualitatively worse than bigger ones, but they are faster, need less memory and might already be sufficient for your use case.

High-level explanation

You can use any program that creates a server with OpenAI-compatible API.

After you started your service, you can do this:

  1. The "Chat Model" field in AI preferences is editable, so you can enter any model you have downloaded

  2. There is a field called "API base URL" in "Expert Settings" where you need to provide the address of an OpenAI-compatible API server

VoilĂ ! You can use a local LLM right away in JabRef.

Step-by-step guide for ollama

The following steps guide you on how to use ollama to download and run local LLMs.

  1. Install ollama from

  2. Select a model that you want to run. The ollama provides to choose from (we recommend trying , or , or )

  3. When you have selected your model, type ollama pull <MODEL>:<PARAMETERS> in your terminal. <MODEL> refers to the model name like gemma2 or mistral, and <PARAMETERS> refers to parameters count like 2b or 9b.

  4. ollama will download the model for you

  5. After that, you can run ollama serve to start a local web server. This server will accept requests and respond with LLM output. Note: The ollama server may already be running, so do not be alarmed by a cannot bind error. If it is not yet running, use the following command: ollama run <MODEL>:<PARAMETERS>

  6. Go to JabRef Preferences -> AI

  7. Set the "AI provider" to "OpenAI"

  8. Set the "Chat Model" to the model you have downloaded in the format <MODEL>:<PARAMETERS>

  9. Set the "API base URL" in "Expert Settings" to http://localhost:11434/v1/

Now, you are all set and can chat "locally".

Step-by-step guide for GPT4All

The following steps guide you on how to use GPT4Allto download and run local LLMs.

  1. Open JabRef, go to "File" > "Preferences" > "AI"

  2. Set the "AI provider" to "GPT4All"

  3. Set the "Chat model" to the name (including the .ggufpart) of the model you have downloaded in GPT4All.

  4. Set the "API base URL" in "Expert Settings" to http://localhost:4891/v1/chat/completions.

Install GPT4Allfrom their .

Open GPT4All, , configure it in the and .

their website
a large list of models
gemma2:2b
mistral:7b
tinyllama
website
download a model
settings
run it as a server