100% private LLMs (GPT) with Practicus AI v23.11

Private LLM

We are pleased to announce that today we released Practicus AI v23.11 offering several new features, including the ability to use private Large Language Models (LLMs) as-is, or fine tune, RAG, and deploy custom ones to meet your specialized needs. Please continue reading to learn more.

Big Picture

Practicus AI was already allowing you to use LLMs such as OpenAI ChatGPT to analyze and prepare data. We now allow you to quickly use your private LLM so that you can:

  • Customize and fine tune LLMs to better meet your needs, the most flexible way possible.
  • Protect your privacy and data by not sharing it with third parties.

In a nutshell, we offer you 3 options to take advantage of LLMs.

Option 1 – Public Cloud & Public LLM.  (Business As Usual)

You can connect Practicus AI to OpenAI ChatGPT, Google Bard or similar online services. You can use your own private accounts and access keys for these public services.

Option 2 – Public Cloud & Private LLM. 

In this scenario, Practicus AI helps you use open source LLMs as-is, or build customized ones,  and run them in public cloud infrastructure providers such as AWS, Azure and GCP.

Option 3 – Private (Edge) Cloud & Private LLM.

This is the strictest deployment scenario where Practicus AI, LLMs and all other components run air-gapped without an exception. (100% disconnected from the internet).

Now let’s take a look at some of the benefits.

Benefit #1 - 100% private co-pilot

The first (and quickest to obtain) benefit of private LLM is to offer an alternative to public services such OpenAI ChatGPT or Google Bard. Practicus AI offers, with one click, to host open LLM models such as Meta’s LLAMA 2 or  TII’s Falcon. 

Click to open GPT in "co-pilot" mode
Ask any complex question to analyze or prepare your data
System admin decides on the privacy and security requirements
End user gets the result, keeps company data private

Benefit #2 - Custom LLM development and hosting

Benefits private LLMs go way beyond the core use case of Practicus AI, Advanced Analytics. By taking advantage of our GPU optimized MLOps, you can host your LLM models for use cases related to call centers, customer support chatbot systems, internal knowledge base and support systems and many more. 

Data Scientist requests (by clicking or coding) an 'LLM ready' worker

You can use Practicus AI container images with LLM development features such as Nvidia CUDA, TORCH etc, or build your own images from scratch, or by basing them from our library.

Data Scientist fine-tunes and / or implements RAG, 
and then publishes the 'private' model with 'one line of code'

Data scientists can now focus 100% on model development, and 0% on infrastructure, CUDA libraries, hosting, security, compliance, governance …

MLOps Admin changes where the model runs, tweaks capacity, security 
and other configuration

MLOps Admin:  “We have 3 active model versions running, let me switch the new version to run on Nvidia H100 GPUs, but only with 10% of the request traffic. Active traffic should not be effected”

Note: An MLOps engineer can also choose to code in .yaml instead of using the admin GUI. They all lead to the same automated GitOps CD system for operational excellence and repeatability.

Internal or external developers securely consume private LLM APIs

Your internal organization developers or your partners can now call your private LLM API endpoints, and integrate the wisdom to various systems.

Please feel free to download Practicus AI for free and get started right away.

Thank you!

Comments are closed.