Computer

Large Language Model

search

Large Language Model, Generative AI, Agentic AI

  • Precautions
  1. Non-Medical
    1. Not for Medical Care
    2. The author uses several software packages and programming languages to develop FPnotebook content
    3. For convenience, a dozen pages in FPNotebook are dedicated to quick notes on content creation
  2. Quality and Safe Usage
    1. Generative AI is imperfect and requires human oversight, evaluation and iterative improvement
    2. Remember: LLMs generate answers by stringing together the statistically most likely next word
      1. The answer may not be correct, complete, legal, ethical or responsible
      2. Always operate with guard rails and oversight
  • Types
  • LLM Models
  1. General
    1. Text data passed to LLMs is tokenized
      1. Words are broken into common segments (~4 chars each)
      2. Numbers are broken into groups of 3 numbers each
    2. Large frontier models typically require payment in the form of price per million tokens
      1. Cost is based on the combination of input tokens, compute tokens and output tokens
  2. Large Frontier Models (run in the cloud, 100s of billions to trillions or parameters)
    1. OpenAI GPT
    2. Anthropic Claude
    3. Google Gemini
    4. Deepseek
    5. Grok
  3. Smaller Models (may be run locally with Ollama, model variants with<10 Billion parameters)
    1. Meta LLama
    2. Mistral
    3. Microsoft Phi-3
    4. Alibaba Qwen
    5. Google Gemma
    6. Deepseek
  4. Resources
    1. Vellum Leaderboard
      1. https://www.vellum.ai/llm-leaderboard
  • Types
  • LLM Usage Tiers
  1. Level 1: Transformers (multi-shot prompting)
    1. Chat with a model via Model Specific Web Interface (e.g. Chat GPT)
    2. Chat with a model via Consolidator (e.g. Perplexity)
    3. Chat with a model via API (e.g. python in Jupyter Notebook)
      1. Create json with an array of dictionary objects (contains entire history of chat)
        1. role: system, user, assistant, tool, developer
        2. content: message
  2. Level 2: Agents (one-shot prompting)
    1. Give a model extensive system instructions to be used with every query
    2. Give the agent access to a personal or business knowledge base in various forms
      1. Data may be vectorized for rapid access and context (Retrieval-Augmented Generation or RAG)
    3. Agent masters domain and task specific queries
  3. Level 3: Agentic AI Workflows (Virtual Assistants)
    1. Agents are used in various combinations with one another to form an overall workflow
      1. Agents may be used in serial or parallel, with data passed along the chain of agents
      2. Workflow may include if-then branching logic
      3. Workflows may be autonomous, but typically involve human oversight
    2. Interfaces
      1. Web Interfaces (e.g. Cassidy, TypingMind)
      2. Coded Interfaces (e.g. python)
  • Techniques
  • Prompting Pearls
  1. Prompting instructions to ensure quality Generative AI answers
    1. Stop and Think deeply about the question
    2. Think Step by Step
    3. If you do not know the answer, say so (do NOT hallucinate)
  2. Recursive use of AI Transformers and Agents
    1. Prompt: "What questions should I answer so that you can generate the best answer to my question?"
    2. Prompt: "I asked another model the question X, and they responded with answer Y. Evaluate their answer."
  • Technique
  • System Instructions for Agent Development
  1. Have AI chat generate the system instructions for you
    1. Give it the agents general tasks to reach target output (ultimate goal for the agent)
    2. Ask the AI chat to ask you 5-10 questions related to the agent development it needs to start
    3. Answer the questions, and any follow-up questions
    4. Have the AI chat generate a draft of the system instructions into fields as shown below
  2. System instruction fields (in Markdown for ChatGPT or XML for Claude)
    1. Role
      1. Assistant primary function, expertise and personality
    2. Context
      1. Background information (business info, common scenarios, company values)
    3. Instructions
      1. Step-by-step guides and protocols
      2. Address general and specific scenarios
    4. Criteria
      1. Benchmarks on which to grade the quality of the output
      2. Identify strict rules (Do's and Don'ts)
    5. Examples
      1. Sample scenarios (at least 2-3) with their expected outputs
  3. References
    1. Tyler Fisk, Maven Course: Agentic AI Foundations