The answer, for the agent I built, was yes about half the time. I scored 50% on the GAIA Level 1 evaluation set. The pass threshold for the certificate is 30%. That gap between 30% and 50% is not random. It is a direct product of architectural choices. This article explains what those choices were, what the code does, and what you need to understand if you want to replicate or improve on that result.

What GAIA Actually Tests

Before discussing the agent, it is worth being precise about what GAIA is and why it is a credible benchmark.

GAIA is a benchmark designed to evaluate AI assistants on real-world tasks that require a combination of core capabilities, including reasoning, multimodal understanding, web browsing, and proficient tool use. It features 466 carefully curated questions that are conceptually simple for humans yet remarkably challenging for current AI systems. Humans achieve roughly 92% success. GPT-4 with plugins managed around 15%. OpenAI’s Deep Research reached 67.36% on the validation set.

That spread is informative. The difficulty is not linguistic. The questions are not ambiguous or poorly formed. The difficulty is structural. Answering correctly requires an agent to plan across multiple steps, call the right tools in the right order, handle file attachments, parse web content, and return an exact string match. There is no partial credit. Either the answer matches the ground truth precisely or it does not.

GAIA is organised into three levels. Level 1 requires fewer than five steps and minimal tool use. Level 2 involves more complex reasoning and coordination between multiple tools across five to ten steps. Level 3 demands long-term planning and advanced tool integration.

The course evaluation uses 20 questions drawn from Level 1. That is the correct starting point. Level 1 is tractable. It is also honest about where the field currently sits.

The Architecture: Three Files, One Deployed Space

The agent I submitted lives in a public Hugging Face Space, not a GitHub repository. This matters because the course API requires a public code link ending in /tree/main pointing to your Space. The Space is the product. It is what gets evaluated.

The implementation consists of three files: app.py, requirements.txt, and README.md. That is it. There is no fourth file hiding complexity. The simplicity is intentional.

app.py: Where the Reasoning Happens

The core of the system is a smolagents.CodeAgent backed by Claude Sonnet 3.5, accessed through LiteLLM. The decision to use LiteLLM as the model interface rather than calling the Anthropic API directly gives you flexibility: the same code works with any OpenAI-compatible endpoint by changing a single string. Claude Sonnet was chosen over smaller open-source models for a direct reason. GAIA Level 1 questions require multi-step reasoning in a single pass. Smaller models accumulate errors across steps. The quality of the underlying model is not a peripheral consideration. It is the primary determinant of ceiling performance.

The agent receives two tools. The first is DuckDuckGo web search, a standard component in most smolagents setups. When a question references a fact that changes over time, or one that a language model could not plausibly have memorised, the agent searches for it rather than confabulating. The second tool is a custom file reader. GAIA tasks frequently include attached files: spreadsheets, PDFs, text documents. Without the ability to read those files, a significant fraction of questions are unanswerable regardless of how well the agent reasons.

Here is what app.py does at a structural level. It authenticates with Hugging Face using the Gradio OAuth flow, fetches the list of evaluation questions from the course API, downloads any associated files for each question, runs the agent against each question, collects the answers, and posts them to the submission endpoint. The scoring is exact match. The prompt instructs the agent to return only the answer, with no preamble, no explanation, and no trailing text. That instruction is critical. The evaluation system does not strip formatting. If your agent writes “The answer is 1492” instead of “1492”, the answer is marked wrong.

The CodeAgent architecture generates Python code at each reasoning step rather than issuing JSON tool calls. This means the agent can write small programs to manipulate retrieved data. If a question asks for a sorted list derived from a spreadsheet, the agent can write a sort and return the result. This is more powerful than a tool-calling architecture for questions that require data transformation, not just data retrieval.

requirements.txt and README.md: The Operational Layer

requirements.txt specifies the exact package versions needed for the Space to build reliably. The key dependencies are smolagents, litellm, anthropic, duckduckgo-search, and gradio. Version pinning matters in production. HuggingFace Spaces rebuild from scratch when you push changes. An unpinned dependency that updates overnight can break your submission pipeline.

README.md includes the Spaces metadata header, which must set sdk: gradio and hf_oauth: true. The OAuth flag is what enables the Gradio login button, which in turn provides your HuggingFace username to the submission API. Without it, the API cannot record who submitted the answers.

The ANTHROPIC_API_KEY is stored as a Space secret, not in the code. It is injected as an environment variable at runtime. You set this once in the Space settings. It is never visible in the public repository. This is not just good practice. It is mandatory. A key committed to a public Space is a key that will be rotated by Anthropic’s automated systems within hours.

The 50% Score: What Made the Difference

The pass threshold is 30%, which corresponds to six correct answers out of twenty. I scored 50%, which is ten correct. The difference between a mediocre setup and a solid one on GAIA Level 1 is not primarily about the framework. It is about three specific decisions.

The first is model selection. Claude Sonnet reasons reliably across five or more sequential steps without losing track of the original goal. Many students attempting this unit with smaller open-source models report scores in the 20 to 35 percent range not because their tool setup was wrong, but because the model failed midway through a reasoning chain and either looped or gave up.

The second is the file reader tool. Roughly a third of GAIA Level 1 questions include an attached file. An agent without file-reading capability is effectively forfeiting those questions. Adding a tool that reads text, CSV, or basic document formats and returns the content as a string is a low-effort, high-return improvement.

The third is the system prompt. The exact-match scoring penalises agents that are verbose. Every agent needs an explicit instruction to answer with the bare minimum: no “Based on my research...”, no “The answer is...”, just the answer. This is not obvious if you have been building agents for conversational use cases. In a conversational context, verbosity reads as helpfulness. In a benchmarked evaluation, it reads as a wrong answer.

Where the 50% Did Not Come From

Intellectual honesty requires addressing what the agent got wrong. Multi-hop retrieval failures account for a significant portion of the missed questions. GAIA Level 1, while simpler than Levels 2 and 3, still includes questions that require finding fact A, using fact A to look up fact B, and then performing a transformation on fact B. If the web search at step one returns a page that partially answers the question, the agent sometimes treats that partial answer as complete and moves to the submission step too early. This is not a tool failure. It is a planning failure. The agent does not have a strong enough internal check for whether it has actually resolved the original question.

Multimodal questions were also a consistent failure point. Some GAIA tasks include images that must be analysed as part of the answer. The current setup does not pass image content to the model. Those questions were effectively unresolvable with this architecture.

Students aiming for scores above 50% on this evaluation should focus on those two failure modes. Better replanning logic, periodic self-checks against the original question, and vision-capable model integration are the most direct paths to improvement. This is speculation based on observed failure patterns, not a guaranteed prescription.

Deploying on a HuggingFace Space

The deployment process is deliberate. You create a new Space, set the SDK to Gradio in the Space creation form, and add hf_oauth: true to the README metadata. You upload the three files. You add the Anthropic API key as a secret in the Space settings. The Space builds automatically.

Once the Space is live, you open it in a browser, log in with your HuggingFace account, and click the evaluation button. The app runs the agent against all twenty questions, collects the answers, and submits them. The course API scores the submission and updates the student leaderboard. Your Space code link is attached to your leaderboard entry. The leaderboard requires the Space to remain public. If you make it private, the submission is effectively unverifiable and may be removed.

The entire flow, from a blank Space to a scored submission, takes under an hour if the Space builds cleanly. If you hit dependency conflicts, check your Python version assumption. HuggingFace Spaces default to a recent Python version. If smolagents or LiteLLM has a minimum version requirement, pin explicitly.

What Unit 4 Actually Teaches

The GAIA benchmark is valuable not because it is hard to game, though it is, but because it measures something real. Correct answers demand full task execution, making brute-forcing ineffective. Answers are concise, factual, and unambiguous, which is ideal for benchmarking.

What it measures is the gap between a language model that can discuss a topic and an agent that can resolve a task. That gap is larger than most practitioners expect. The course sequence up to this point has built each piece: action representation in Unit 2, retrieval in Unit 3. Unit 4 places those pieces under adversarial conditions. The questions do not cooperate. The files are real. The web search returns noise alongside signal.

The result is an honest accounting of where agent technology sits today. Capable, narrow, and dependent on strong underlying models and precise tool design. The 50% ceiling is not a failure of the framework. It reflects the genuine difficulty of reliable multi-step reasoning under uncertainty.

If you are working through the course and want to see where you stand, the student leaderboard is public and updated in real time at Student Board. My certificate submission, along with those of other students who cleared the threshold, is visible in the course discussion at Certificate. Both links are worth checking before you submit. The leaderboard shows what scores are actually achievable with different setups, and the discussion thread contains honest accounts from students who cleared 30%, 50%, and above. That is more useful than any tutorial.

The code is hosted publicly at my HuggingFace Space. It is three files. Read them before you write your own.

Units 2.1 through 2.3 examined how agents act, how they represent action as code or structured calls, and how execution flow can be controlled by the developer rather than delegated entirely to the model. Unit 3 changes the question. It stops asking how agents act and starts asking what agents know. The answer it builds toward is not optimistic. Agents, by default, know only what was in their training data. That data has a cutoff date. It contains no record of your guests, your private documents, or your organisation’s internal state. Retrieval Augmented Generation exists precisely to address this structural limitation.

What follows is not a walkthrough of the documentation. It is an account of what the code we built reveals about the architecture of memory, retrieval, and tool composition in agent systems, explained as plainly as possible.

The Problem the Course Sets Up

The course frames the unit around a gala. Alfred, the agent, is hosting a party and needs to answer questions about guests, weather conditions, and the professional work of AI researchers attending the event. None of this information lives inside a language model. Guest names, email addresses, and personal anecdotes are bespoke data. The model cannot recall what it was never shown.

This is not a hypothetical limitation. It is a mathematical one. A language model is a function trained on a fixed corpus. Its parameters encode statistical relationships across tokens in that corpus. Information that never appeared in the corpus does not influence any parameter. Asking the model to recall it produces not silence but confident fabrication, because the model has no mechanism to distinguish between knowing something and generating something plausible.

RAG solves this by separating storage from reasoning. You keep the information in a retrieval system. When a question arrives, you fetch the relevant records first, then pass them to the model as context. The model reasons over retrieved content rather than over its own compressed memory. The agent variant, Agentic RAG, goes one step further. Rather than forcing retrieval to happen on every query, the agent decides when retrieval is necessary. It treats the retrieval system as one tool among several, calling it only when the question warrants it.

That is the architecture this unit builds. The five files we wrote during the session implement it from the ground up.

guest_data: The Filing Cabinet

Think of guest_data as building a filing cabinet. You start with a raw table of information: each guest has a name, a relation to the host, a description, and an email address. That table, pulled from the Hugging Face dataset agents-course/unit3-invitees, is structured for storage but not for search.

The code converts each row into a Document object. A Document is nothing more than a text string plus a small dictionary of metadata. The text string concatenates all the guest fields into a single readable block. The metadata stores the guest’s name separately so it can be used for filtering later.

This step is not technically sophisticated, but it is architecturally important. You are making a decision about how information will be represented before retrieval. Every decision made here affects what the retriever can find. If you omit a field from the text block, the retriever cannot match against it. If you format the fields inconsistently, keyword matching degrades. The quality of what the agent can retrieve is bounded by the quality of what you put into the filing cabinet.

In Feynman terms: imagine writing index cards for every guest. On each card you write their name, how you know them, a short story about them, and their contact details. The next step is building a system to find the right card when someone asks a question.

retriever: The Librarian

The retriever file builds the search mechanism on top of the filing cabinet. The specific algorithm used is BM25, which stands for Best Match 25. You do not need to memorise the name. What matters is what it does.

BM25 is a keyword scoring function. Given a query and a collection of documents, it scores each document by how well its words match the query words, with corrections applied for document length and word frequency. A document that contains the query term ten times is not ten times as relevant as one that contains it once. BM25 applies a diminishing return to term frequency and a normalisation factor for length, so short documents with one precise match can outscore long documents with many incidental mentions.

The practical implication is that BM25 requires no machine learning model to run. There are no embeddings, no vector database, no GPU. It is a deterministic function over word counts. This makes it fast, cheap, and reproducible. The tradeoff is that it matches words, not meanings. A query for “mathematician” will not match a document that says “logician” unless those two words appear together somewhere in the corpus.

The GuestInfoRetrieverTool wraps BM25 in the smolagents Tool interface. This wrapper is what makes the retriever visible to the agent. The name and description fields are not documentation for human readers. They are the signal the agent uses to decide whether to call this tool at all. If the description is vague, the agent may call the wrong tool or skip retrieval entirely. Precise descriptions are not a nicety. They are load-bearing.

The forward method takes a query string, runs it through the BM25 retriever, and returns the top three matching documents as a single formatted string. The agent receives this string as the result of a tool call and incorporates it into its next reasoning step.

In Feynman terms: the retriever is a very fast librarian who cannot read, but can match words. You hand her a note that says “Ada Lovelace” and she finds every index card with those words on it, sorted by how prominently those words appear. She hands back the top three cards. She never interprets. She only matches.

tools: The Specialist Staff

The tools file adds three capabilities that have nothing to do with the guest list. This is where the architecture reveals something important about how agentic systems differ from simple retrieval pipelines.

A plain RAG system has one retrieval path. A query arrives, retrieval runs, the model answers. The retriever and the query are tightly coupled. If the question is not about the documents, the system fails gracefully or produces a hallucination because it has nothing else to offer.

An agentic RAG system has multiple retrieval paths. The agent chooses which one to invoke. The tools file equips Alfred with three additional tools: a web search tool using DuckDuckGo for current events, a weather tool that calls an external API for location-specific forecasts, and a Hub stats tool that queries the Hugging Face Hub for model download statistics.

Each of these tools is a separate class that inherits from the Tool base class in smolagents. Each defines its name, description, inputs, and a forward method. The pattern is identical for every tool. This is intentional. The framework enforces a consistent interface so the agent can treat all tools symmetrically. From the agent’s perspective, calling the weather tool and calling the guest retriever are the same operation: provide a string input, receive a string output, reason over the result.

The weather tool in this unit uses dummy data. It returns a randomly selected condition from a hardcoded list. This is an honest simplification. The course signals clearly that a production implementation would call the OpenWeatherMap API or equivalent. The structure of the tool does not change when you swap the data source. Only the forward method changes.

The Hub stats tool calls a real API. Given an author name, it queries the Hugging Face Hub for that author’s most downloaded model and returns the model name and download count. This is a live call against production infrastructure. The tool demonstrates that agents can interact with external services in real time, not just retrieve from static documents.

In Feynman terms: imagine Alfred has three specialist consultants on speed dial. One searches the internet. One calls a weather station. One queries a database of AI model statistics. Alfred decides who to call based on the question. The guest list lives in a separate system. These consultants handle everything outside it.

app: Assembling Alfred

The app file is where the components come together. It initialises the model, instantiates each tool, and creates the agent by passing all four tools into a CodeAgent.

The CodeAgent class in smolagents uses a model to generate and execute Python code at each reasoning step. This is the same architecture examined in Unit 2.1. The agent does not produce JSON function calls. It writes Python code that calls the tools it needs, executes that code, observes the output, and decides what to do next. The code is ephemeral. Each step produces new code based on the current state of the reasoning trace.

The planning_interval=3 parameter instructs the agent to produce an explicit plan every three steps. This is a concession to the context window. Without periodic replanning, the agent’s reasoning trace can drift as the number of steps grows. Forcing a planning step introduces a checkpoint where the agent re-evaluates its progress against the original goal. This reduces the probability of the agent pursuing a locally coherent but globally irrelevant sequence of tool calls.

The add_base_tools=True parameter adds the framework’s built-in tools to Alfred’s toolkit alongside the custom ones. These include a Python code executor and a basic file reader. They are supplemental. The four custom tools carry the actual functional weight.

In Feynman terms: app.py is the moment Alfred walks into the room fully dressed. Before this file, you had a wardrobe full of separate items: a filing system, a librarian, three specialist consultants, and a reasoning engine. This file puts them all together and says: Alfred, here is everything you need, now answer questions.

app_with_memory: The Follow-Up Conversation

The app_with_memory component addresses a structural limitation in the base agent. By default, every time you call alfred.run(), the agent begins with a blank slate. It has no knowledge of what it found in the previous call. If you ask “Tell me about Ada Lovelace” and then follow up with “What projects is she currently working on?”, the second call does not know who “she” refers to.

The fix in smolagents is a single parameter: reset=False. Passing this to alfred.run() on subsequent calls instructs the agent to retain its message history from previous calls rather than clearing it. The conversation accumulates. The agent can refer back to what it found, what tools it called, and what conclusions it reached.

This is not the same as persistent memory. The history lives in the process. If you restart the server, it disappears. For a gala use case, this is sufficient. Alfred needs to remember what he discussed with you five minutes ago, not what he discussed at last year’s gala. For production systems requiring true persistence, the message history must be serialised and stored externally between sessions.

The course notes, correctly, that memory in agentic frameworks is not a default. smolagents requires the explicit reset=False flag. LlamaIndex requires a context object to be passed. LangGraph offers a MemorySaver component that checkpoints state to a store. All three frameworks treat memory as opt-in because stateless execution is easier to reason about, debug, and test. The choice to add memory is a deliberate architectural decision, not a configuration option to tick by default.

In Feynman terms: without memory, each time you speak to Alfred he has just woken up from dreamless sleep. He is polite and capable but has no idea who you spoke about a moment ago. With reset=False, Alfred remembers the thread. He can refer to Ada by name in the next sentence because he recalls mentioning her in the last one.

What the Unit Reveals About the Architecture of Agent Knowledge

The five files in this unit implement a clear separation of concerns. Guest data is stored as documents. The retriever indexes those documents and exposes search as a callable tool. The additional tools expose external services through the same interface. The agent receives all tools with equal standing and decides which to use based on the question. Memory, when enabled, allows multi-turn reasoning over a persistent context.

This separation is not ornamental. It reflects a genuine architectural truth. An agent’s knowledge has two components: parametric knowledge, which is whatever the model absorbed during training and cannot be changed without retraining, and contextual knowledge, which is whatever you inject at inference time through retrieval, tool calls, and conversation history. Agentic RAG is a framework for maximising the quality and relevance of contextual knowledge.

The limitation the course does not fully address is precision. BM25 retrieves by keyword match. If the guest’s description uses different vocabulary to the query, retrieval can fail silently. The agent receives an empty result, and the model, faced with nothing concrete, may fall back on parametric knowledge, which is where fabrication begins. For higher-stakes applications, embedding-based retrieval or hybrid approaches that combine keyword and semantic search are more robust. The course acknowledges this and points toward sentence-transformers as a natural next step.

That is an honest position. BM25 is a sound baseline. It is not the ceiling.

All corrected notebooks with documented observations are available in our GitHub repository. Each correction includes a comment explaining the root cause and the reasoning behind the fix.

What follows is not a summary of the documentation. It is an account of what the material reveals about orchestration philosophy, state management, and the control-versus-freedom tradeoff in agent architecture when read with an engineering perspective.

One practical note before the substance: we ran the notebooks in this unit end to end, as we did in the previous two. We encountered one error that is not addressed in the course material, which I document in the errors section below.

The Architectural Position LangGraph Occupies

Every agent framework must answer the same question: who decides what happens next, the developer or the model?

smolagents answers clearly. The model decides. It writes Python code, calls tools in whatever order it determines, and the framework executes what it produces. The developer defines tools and sets constraints, but the model controls the sequence. LlamaIndex occupies a middle position. Its workflow system lets the developer define steps and event types, but agents within those steps retain autonomy over tool selection and reasoning.

LangGraph sits at the opposite end of this spectrum. The developer designs a directed graph of nodes and edges. The graph defines every possible path the execution can take. The model operates within individual nodes, but the transitions between nodes are specified in code. This is not a subtle difference. It is a fundamentally different philosophy about where intelligence should reside in an agentic system.

The course frames this as a tradeoff between “control” and “freedom.” That framing is correct, but it understates the engineering implications. Freedom means the model can adapt to unforeseen situations. Control means the developer can guarantee that certain states are always reached, that certain checks always occur, and that the system behaves predictably under audit. For production systems that handle sensitive data, interact with customers, or make decisions with legal consequences, control is not a preference. It is a requirement.

The Three Primitives: State, Nodes, Edges

LangGraph’s entire abstraction rests on three concepts. Understanding them is sufficient to build any application the framework supports.

State is a user-defined TypedDict that carries all information through the graph. Every node receives the current state and returns updates to it. This is the most consequential design decision in the framework, because the state definition determines what the graph can reason about. If a piece of information is not in the state, no node can access it. If a field is poorly typed or ambiguously named, every node that touches it inherits that ambiguity.

The course demonstrates this with an EmailState that tracks the email being processed, its classification, spam reasoning, a draft response, and the full message history with the LLM. This is a well-structured example precisely because every field serves a clear purpose in at least one node. In practice, state design is where most LangGraph applications succeed or fail. A bloated state that tracks irrelevant information slows every node. A lean state that omits a critical field forces you to restructure the entire graph when you discover the omission.

One detail worth noting: LangGraph supports annotated state fields with custom operators. The add_messages operator, demonstrated in the document analysis example, appends new messages to the list rather than overwriting it. This is a meaningful feature for any application where multiple nodes contribute to a conversation history. Without it, every node would need to manually concatenate its messages with the existing list, which is error-prone and verbose.

Nodes are Python functions. Each receives the state, performs some operation, and returns a dictionary of state updates. The framework merges these updates into the current state before passing it to the next node. Nodes can contain LLM calls, tool invocations, deterministic logic, or side effects like printing or logging.

The simplicity of this interface is LangGraph’s most under-appreciated advantage. A node is just a function. It can be unit-tested independently of the graph. It can be replaced without modifying adjacent nodes. It can be reused across different graphs. This composability is not theoretical. It is the direct consequence of a clean functional interface that takes state in and returns state updates out.

Edges define transitions between nodes. There are two types. Regular edges connect one node unconditionally to another. Conditional edges invoke a routing function that inspects the current state and returns a string indicating which node to visit next. The routing function’s return value must match one of the keys in a mapping provided to add_conditional_edges.

This is where the control philosophy manifests concretely. In smolagents, the model decides the next action at every step. In LangGraph, the developer writes a routing function that makes that decision based on state. The routing function can be as simple as checking a boolean field, or as complex as evaluating multiple conditions across several state fields. The point is that the logic is explicit, inspectable, and testable. It does not depend on the model’s judgement at transition time.

The Email Classification Graph: Structured Workflow Without Agency

The first practical example in the course builds an email processing system. It reads an email, classifies it as spam or legitimate using an LLM, then either discards it or drafts a response. The course is explicit that this is not an agent. No tool calling is involved. The purpose is to demonstrate LangGraph’s orchestration primitives in isolation from agentic complexity.

This is a sound pedagogical choice. The graph has five nodes: read_email, classify_email, handle_spam, draft_response, and notify_mr_hugg. It has one conditional edge after classification that routes to either the spam handler or the response drafter based on the is_spam field in the state.

The classification node is worth examining closely. It constructs a prompt, calls the LLM, and parses the response with string matching to determine whether the email is spam. The parsing logic checks whether “spam” appears in the response and “not spam” does not. This is fragile. A response containing “this is not spam” would be correctly handled, but a response containing “I considered whether this might be spam, but it is legitimate” would be misclassified. In production, you would replace this with structured output parsing or a dedicated classification call with constrained output format.

The graph construction follows a consistent pattern: instantiate StateGraph with the state type, add nodes, add edges from START to the first node, add regular and conditional edges between nodes, and terminate paths with END. The compile() call produces an executable graph. This pattern is identical for every LangGraph application regardless of complexity. Once you internalise it, you can read any LangGraph codebase without additional explanation.

The course also introduces Langfuse for observability. The integration is a single callback handler passed to compiled_graph.invoke(). This is operationally important because LangGraph applications with multiple conditional paths are difficult to debug from logs alone. A tracing tool that shows which nodes executed, in what order, and what state transformations occurred at each step is not optional for production deployments. It is necessary infrastructure.

The Document Analysis Agent: ReAct Within a Graph

The second example introduces genuine agency. It builds a document analysis system where the model can call tools: a vision model for text extraction from images, and a division function for calculations. The architecture follows the ReAct pattern, where the model reasons about what tool to call, calls it, observes the result, and decides whether to continue.

[INSERT IMAGE: Alfred’s Document Analysis Workflow]

The critical difference from the email classifier is the loop. After the assistant node produces a tool call, the graph routes to a tools node that executes the call, then routes back to the assistant node. This cycle continues until the model produces a response that is not a tool call, at which point tools_condition routes to END.

This is where LangGraph’s graph structure earns its value. The loop is not implicit. It is an explicit edge from tools back to assistant. The termination condition is not hidden inside the model’s reasoning. It is a named function, tools_condition, that the developer can inspect, test, and replace. If you wanted to add a maximum iteration limit, you would modify the routing function to check a counter in the state. If you wanted to add a human approval step before certain tool calls, you would add a node between assistant and tools with a conditional edge.

The ToolNode class from LangGraph’s prebuilt module handles tool execution. You pass it a list of tools, and it automatically dispatches calls based on the model’s output. The tools_condition function inspects the model’s last message and returns either "tools" if a tool call is present or routes to END if not. These are convenience abstractions, but they encode a specific pattern: the ReAct loop as a two-node cycle with a conditional exit.

The state in this example uses Annotated[list[AnyMessage], add_messages], which means the message list accumulates across all nodes rather than being overwritten. This is essential for the ReAct pattern because the model needs to see the full history of its reasoning, tool calls, and tool results to make informed decisions about what to do next.

Where We Found Errors in the Notebooks

We found one error when running the notebooks locally that the course does not address. It does not appear in every environment, but when it does, it produces alarming output that obscures the actual results.

Asyncio task destruction warning in Jupyter when invoking the LangGraph agent. When running the document analysis agent notebook locally in a Jupyter environment, the cell that calls react_graph.invoke() completes its execution and returns the correct result, but immediately afterwards the kernel emits a cascade of warnings:

Task was destroyed but it is pending!
task: <Task pending name='Task-190' coro=<_async_in_context.<locals>.run_in_context() done,
defined at .../ipykernel/utils.py:57> wait_for=<Task pending name='Task-191'
coro=<Kernel.shell_main() running at .../ipykernel/kernelbase.py:597>
cb=[Task.task_wakeup()]> cb=[ZMQStream._run_callback.<locals>._log_error() ...]>

/Users/.../.pyenv/versions/3.12.4/lib/python3.12/typing.py:407: RuntimeWarning:
coroutine 'Kernel.shell_main' was never awaited

RuntimeWarning: Enable tracemalloc to get the object allocation traceback

Task was destroyed but it is pending!
task: <Task pending name='Task-191' coro=<Kernel.shell_main() running at
.../ipykernel/kernelbase.py:597> cb=[Task.task_wakeup()]>

This error looks severe but is not. The root cause is a conflict between LangGraph’s internal async execution and Jupyter’s own event loop. LangGraph uses asyncio internally even when you call the synchronous .invoke() method. It creates a new event loop, runs the graph, and tears the loop down. When this happens inside Jupyter, which already has its own running event loop managed by ipykernel, the teardown of LangGraph’s temporary loop can orphan pending tasks that belong to the Jupyter kernel itself. The kernel continues to function, but the Python runtime emits warnings about tasks that were destroyed before completion.

The fix requires one of two approaches. The first and simplest is to use nest_asyncio, which patches Python’s event loop to allow nested execution:

import nest_asyncio
nest_asyncio.apply()

# Then invoke normally
result = react_graph.invoke({"messages": messages, "input_file": None})

The second approach, more appropriate for production code, is to use the async interface directly:

result = await react_graph.ainvoke({"messages": messages, "input_file": None})

This works because ainvoke cooperates with the existing Jupyter event loop rather than creating a competing one. The course uses .invoke() throughout, which is the correct default for scripts but can produce these warnings in notebook environments. The warnings are cosmetic in the sense that the computation completes correctly, but they are operationally concerning because they make it difficult to distinguish genuine errors from async lifecycle noise in the notebook output.

We installed nest_asyncio and added the two-line patch at the top of the notebook. The warnings disappeared and all subsequent cells executed cleanly.

What the Course Gets Right

The progression from a non-agentic workflow to an agentic one within the same framework is the strongest aspect of this unit. It demonstrates that LangGraph is not exclusively an agent framework. It is a workflow orchestration framework that can accommodate agents when the application requires them. This distinction matters because many production applications need both: deterministic preprocessing and postprocessing stages surrounding a core agentic loop.

The graph visualisation capability, available through get_graph().draw_mermaid_png(), is more valuable than the course suggests. When a graph has more than five or six nodes with multiple conditional edges, the ability to render it visually and confirm that all paths lead to END is a debugging tool of genuine practical importance.

The course also correctly positions LangGraph as compatible with but independent from LangChain. The LangChain classes (ChatOpenAI, HumanMessage) are used for convenience, but the graph structure itself has no dependency on them. You could replace every LangChain component with raw API calls and the graph would function identically. This separation is important for teams that want LangGraph’s orchestration without committing to LangChain’s abstraction layer for model interaction.

Where LangGraph Sits in the Framework Landscape

Smolagents provides the most freedom. The model writes and executes Python code, choosing its own tools and execution order. This makes it powerful for exploratory tasks where the optimal sequence of actions is unknown at design time. The cost is unpredictability. You cannot guarantee what path the agent will take, and the context window fills with intermediate reasoning that the developer cannot control.

LlamaIndex provides the most complete retrieval infrastructure. Its five-stage pipeline, tool system, and workflow abstraction make it the strongest choice when the core challenge is finding and reasoning over information the model was not trained on. Its workflow system offers structured orchestration, but the framework’s centre of gravity is retrieval, not control flow.

LangGraph provides the most control. Every transition is developer-defined. Every state mutation is explicit. Every conditional branch is a testable function. This makes it the strongest choice when the application has a known structure that must be followed reliably, with LLM reasoning scoped to individual nodes rather than governing the overall flow.

The lesson across all three units is consistent with what we observed in Units 2.1 and 2.2. The binding constraints in agentic systems remain the context window and the boundary between parametric and retrieved knowledge. LangGraph adds a third constraint to the picture: the boundary between deterministic control flow and non-deterministic model reasoning. Its contribution is providing tools to draw that boundary precisely.

No framework is unconditionally superior. The correct choice depends on which constraint dominates your application. If the constraint is compositional computation and dynamic tool selection, smolagents is appropriate. If the constraint is retrieval over private or heterogeneous data, LlamaIndex is appropriate. If the constraint is predictable execution with auditable state transitions, LangGraph is appropriate. In practice, production systems often face all three constraints simultaneously, which is why these frameworks can be composed rather than chosen in isolation.

All corrected notebooks with documented observations are available in our GitHub repository. Each correction includes a comment explaining the root cause and the reasoning behind the fix.

What follows is not a walkthrough of the documentation. It is an account of what the material reveals about retrieval architecture, tool interfaces, and orchestration when you read it with an engineering eye rather than a learning one.

One practical note before the substance: we ran the notebooks in this unit end to end. We found errors, as we did in Unit 2.1. Where relevant, I will point to the specific sections where friction appears and what we did to resolve it.]

The Problem LlamaIndex Is Designed to Solve

It is worth stating the core problem clearly before describing the solution, because the design of LlamaIndex only makes sense against it.

A language model trained on a static corpus knows what it was shown during training and nothing else. It has no access to your internal documentation, your product database, or any document that postdates its training cutoff. When you ask it about those things, it either refuses or create false information. Neither outcome is useful in production.

Retrieval-Augmented Generation, universally abbreviated to RAG, addresses this by separating knowledge storage from inference. Rather than expecting the model to recall information from its parameters, you retrieve relevant content at inference time and inject it into the model’s context. The model reasons over live, scoped, verifiable data rather than parametric memory.

LlamaIndex is built entirely around this idea. It is not a general-purpose agent framework with retrieval bolted on. Retrieval is the first-class concern, and every other capability in the framework, agents, tools, workflows, is designed to be composed with it.

LlamaHub: The Integration Registry

Before any pipeline can be assembled, components need to be installed. LlamaIndex manages this through LlamaHub, a registry of hundreds of integrations covering LLMs, embedding models, vector stores, data loaders, toolspecs, and agents.

The installation convention is consistent across all integrations:

pip install llama-index-{component-type}-{framework-name}

This means the import path mirrors the install command, which reduces the cognitive overhead of locating the correct class for a given integration. The registry is broad enough that most data source integrations you will need already exist in a tested form. This is a genuine operational advantage over frameworks that require custom adapter code for every external service.

The Five-Stage Pipeline as Engineering Discipline

Most frameworks expose retrieval as a single class with a retrieve() method. LlamaIndex exposes it as five discrete, independently configurable stages: loading, indexing, storing, querying, and evaluation.

This decomposition is the framework’s most important design decision, and it is the right one. When a RAG system produces a wrong answer, there are only a finite number of places the failure could have occurred. The document was not loaded correctly. The chunk boundaries cut across relevant context. The embedding model failed to capture semantic similarity. The query strategy returned irrelevant results. The response synthesiser hallucinated beyond what the context supported. Treating these as a single black box makes each failure mode harder to isolate. Treating them as separate stages, with separate abstractions and separate observability, makes each diagnosable.

Loading converts heterogeneous data into uniform Document objects. The entry point for local files is SimpleDirectoryReader, which handles common file types without configuration. For PDFs with complex layouts, LlamaParse provides a managed parsing API, though it is a paid service and that cost is worth noting for teams evaluating it against open alternatives. LlamaHub provides community-built loaders for external sources, databases, and APIs.

Indexing encodes semantic meaning. The IngestionPipeline coordinates two transformations in sequence. First, SentenceSplitter breaks documents into Node objects at natural sentence boundaries. Chunking at fixed character counts is simpler to implement but severs semantic units, and the retrieval quality difference is measurable. Second, each chunk is embedded using a model such as BAAI/bge-small-en-v1.5 through HuggingFaceEmbedding. The embedding converts each chunk into a vector in a shared semantic space where similarity is computable.

One detail that is easy to overlook: the embedding model used during ingestion must be the same model used to embed queries at retrieval time. Using mismatched models produces a retrieval system that works in the sense that it returns results, but the results are not ranked by semantic relevance to the query. It is among the most common sources of degraded retrieval quality in production RAG deployments, and nothing in the framework prevents it unless you enforce the constraint in code.

Storing solves the operational problem of repeated embedding. ChromaDB, integrated through ChromaVectorStore, persists embeddings to disk and makes them reloadable across sessions. The VectorStoreIndex wraps the store and handles all retrieval logic. Once the index is built and persisted, subsequent runs skip the ingestion pipeline entirely.

Querying exposes three interfaces with meaningfully different behaviours. as_retriever returns raw nodes with similarity scores, which is appropriate when you want to inspect what the retrieval layer is actually returning before passing it to a model. as_query_engine produces a synthesised natural language response, which is the appropriate interface for agent tool usage. as_chat_engine maintains conversation history across multiple turns.

The query engine’s internal response strategy is worth understanding because it affects both answer quality and cost. The compact strategy, the default, concatenates retrieved chunks before synthesis and makes a single LLM call. The refine strategy processes each chunk sequentially, updating the answer incrementally, at the cost of one LLM call per chunk. The tree_summarize strategy builds a hierarchical summary and is appropriate when retrieved content is long and heterogeneous. Choosing refine on a corpus that returns ten chunks multiplies your LLM calls by ten. Choosing compact when retrieved chunks are long and varied may cause the model to miss content that falls below its attention in a densely packed context.

Evaluation is where the course earns its credibility. LlamaIndex ships three built-in evaluators: FaithfulnessEvaluator checks whether the response is supported by the retrieved context, AnswerRelevancyEvaluator checks whether the response addresses the original question, and CorrectnessEvaluator checks factual accuracy. All three use an LLM as the judge, which means they inherit the same failure modes as the model being evaluated. They are not ground truth oracles. They are signal, and imperfect signal is more useful than no signal when you are iterating on chunking strategy, embedding models, or prompts.

Tool Design: The QueryEngine as a Structured Interface

Once you have a functioning query engine, the framework provides a clean path to exposing it as an agent tool through QueryEngineTool. You wrap the engine, provide a name and a description, and the agent can call it as part of its reasoning loop.

LlamaIndex provides four tool types in total. FunctionTool wraps any Python function and makes it callable by the agent. The function’s name, type annotations, and docstring are parsed to construct the tool schema. QueryEngineTool wraps a retrieval pipeline as described above. ToolSpecs are curated collections of related tools covering specific external services: the GmailToolSpec, for instance, exposes a coherent set of Gmail operations without requiring you to build the integration yourself. LlamaHub hosts community-maintained toolspecs that cover a growing range of external services.

The description field in every tool type is the operative part. This is the same observation we made about smolagents in the previous unit, and it applies with equal force here. The model selects tools and constructs their arguments based on the descriptions it is given. A vague description produces inconsistent selection. A precise description produces reliable behaviour. Docstrings and descriptions in agentic systems are not documentation for human readers. They are functional specifications that the model executes against.

The framework also integrates with the Model Context Protocol through a McpToolSpec class. You point it at a running MCP server and it exposes that server’s capabilities as tools the agent can use. For teams already building MCP-compatible tooling, this reduces the friction of connecting to external capabilities without writing custom adapters.

Utility tools address a specific operational problem that appears frequently in practice. When a data loader returns a large volume of content, passing all of it to an LLM in a single context is either impossible due to context window limits or expensive due to token volume. The OnDemandToolLoader automatically indexes the content and makes it queryable within a single tool call. The LoadAndSearchToolSpec splits this into two separate tools, one for loading and one for querying, giving the agent more explicit control over the retrieval process.

Agents: Agentic RAG as a Qualitative Change

LlamaIndex supports three agent types. Function Calling Agents use the native tool-calling API of the underlying model. ReAct Agents implement the Reasoning and Acting loop and work with any model that supports text generation. Advanced custom agents are available for complex, domain-specific orchestration.

The practical difference between the first two is worth stating clearly. Function calling agents produce structured tool invocations without requiring the agent to format its reasoning in a specific way. ReAct agents expose their reasoning chain explicitly, showing each thought, action, and observation. For debugging, that visibility is valuable. For token efficiency, it is expensive. The right choice depends on how much you need to understand what the agent is doing at each step versus how much you need to minimise token consumption.

Agents in LlamaIndex are stateless by default. Each run is independent and carries no memory of previous interactions. This is the correct default. Stateless agents are straightforward to reason about, simple to scale, and produce deterministic behaviour given the same inputs. When persistence across interactions is required, a Context object provides it explicitly. The pattern is identical to what smolagents does with its memory management: statelessness as the default, stateful behaviour as an intentional opt-in.

The most substantive capability in this section is agentic RAG. Standard retrieval-augmented generation is a fixed pipeline. A query arrives, a retriever fetches relevant chunks, those chunks are injected into the prompt, and the model generates a response. The model has no influence over how retrieval is performed.

Agentic RAG restructures this by placing retrieval inside the tool loop. The model can formulate its own search queries rather than having the raw user input forwarded to the retriever. It can evaluate what it retrieved and decide whether to search again with a different query. It can route between a local knowledge base and live web search depending on what the question requires. These are not incremental improvements on the standard pipeline. They represent a qualitative change in how information is accessed, and the correct routing depends on the content and intent of individual queries at inference time, not at design time.

Multi-agent systems are composed through the AgentWorkflow class. Each agent receives a name and description. The system maintains a single active speaker at any time, with agents handing off to one another when the task scope falls outside their declared capability. Shared state, such as a counter tracking the total number of function calls across all agents, is propagated through the workflow context and injected into each agent’s prompt through a state_prompt template.

Workflows: Explicit Orchestration Without Implicit Control

The workflow system is the component of LlamaIndex that most clearly distinguishes it architecturally from frameworks that delegate all orchestration to the model.

A workflow in LlamaIndex is a directed graph of steps. Each step is an async function that consumes one or more typed events and emits one or more typed events. The type annotations are the routing mechanism. There is no central dispatcher reading a config file. The framework infers the graph from the type signatures.

This has three practical consequences. First, the workflow graph is statically inspectable. The draw_all_possible_flows utility renders it as an HTML file, which is useful when you are debugging orchestration logic in a system with more than a handful of steps.

Second, data passing between steps is typed. If a step expects a ProcessingEvent and receives a StopEvent, the system catches that mismatch at run time rather than silently producing wrong output. Third, loops and branches are expressed as type unions. A step that can either retry or proceed emits either a LoopEvent or a ProcessingEvent. The control flow is visible in the function signature rather than buried in imperative logic.

The comparison with smolagents is instructive here. smolagents gives the model significant latitude to determine its own execution path. The model decides which tools to call, in what order, and when to stop. The workflow system in LlamaIndex constrains that latitude. The developer defines which steps exist, what events connect them, and what data flows between them. The model reasons within individual steps, but the orchestration is explicit code rather than emergent model behaviour.

Neither approach is unconditionally superior. The smolagents approach is more flexible and requires fewer design decisions upfront. The LlamaIndex workflow approach is more predictable and easier to reason about at the system level. Tasks with well-defined stages and known decision points are a good fit for workflows. Tasks where the agent needs to discover the structure of the problem during execution are a better fit for unconstrained agentic loops.

Looking at your three screenshots, I can identify all four errors precisely. Let me rewrite the errors section to reflect exactly what you found and documented.

Subscribe now

Where we found errors in the notebooks in components and tools

We found five errors across two notebooks in this unit. None of them produce error messages that point clearly to their cause.

First error: package version conflict during installation. The components notebook installs several dependencies in sequence. During this process, the typer package produces a version conflict: the environment already has typer 0.24.1 installed, and the notebook’s requirements force a downgrade to typer 0.11.1. The uninstall and reinstall succeeds technically, but it silently breaks other tools in the environment that depend on the newer version. The fix is to pin a compatible version explicitly before running the rest of the dependency cell, rather than letting the resolver downgrade blindly.

Second error: tracer_provider not defined at force_flush. Later in the same components notebook, a cell attempts to call tracer_provider.force_flush() to flush buffered spans. This produces a clean NameError: name 'tracer_provider' is not defined. The cell assumes that tracer_provider was defined in a previous cell, but the observability setup cell and the query cell are not coupled in the way the notebook implies. If you run the cells out of order, or if the setup cell raises silently, the name never enters scope. The fix is to move the force_flush() call inside the same cell that defines tracer_provider, or to add an explicit guard.

Third error: Arize observability integration does not work as written. The notebook’s observability section uses LlamaTrace with Arize Phoenix through llama-index-callbacks-arize-phoenix. This integration has changed. The correct approach, which we verified, uses the arize.otel register function directly alongside LlamaIndexInstrumentor from openinference-instrumentation-llama-index:

import os
from arize.otel import register
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from dotenv import load_dotenv

load_dotenv(dotenv_path="../.env", override=True)

tracer_provider = register(
    space_id=os.getenv("ARIZE_SPACE_ID"),
    api_key=os.getenv("ARIZE_API_KEY"),
    project_name="my-llamaindex-app",
    batch=False,
    log_to_console=True
)

LlamaIndexInstrumentor().instrument(
    tracer_provider=tracer_provider,
    skip_dep_check=True
)

The notebook contains no indication that the LlamaTrace callback path it documents is outdated. The failure is silent in some configurations and produces a generic connection error in others. Nothing in the error output tells you that the integration interface itself has changed.

Fourth error: get_agent not defined in the tools notebook. The MCP toolspec example in the tools notebook calls a function named get_agent that is never defined in the notebook itself. The cell assumes a locally running MCP server at http://127.0.0.1:8000/sse and a helper function that would connect to it, neither of which exists. The course presents MCP integration as a code snippet without providing the server or the helper. The correct approach is to replace get_agent with an AgentWorkflow constructed directly from the tool list:

from llama_index.core.agent import AgentWorkflow
from llama_index.tools.mcp import BasicMCPClient, McpToolSpec

mcp_client = BasicMCPClient("http://127.0.0.1:8000/sse")
mcp_tool = McpToolSpec(client=mcp_client)
tools = await mcp_tool.to_tool_list_async()

agent = AgentWorkflow.from_tools_or_functions(tools, llm=llm)

You also need a locally running MCP server before any of this executes. The notebook provides neither the server setup nor a substitute for local testing. We documented a minimal server configuration and published it alongside the corrected notebook in our GitHub repository.

Fifth error: agents not rebuilt after context-aware tools are defined in the workflow notebook.

The multi-agent state tracking example in the workflows notebook fails silently. The course shows the add and multiply functions being redefined with a Context parameter, then immediately constructs the AgentWorkflow using multiply_agent and addition_agent that were created earlier in the notebook, before the context-aware versions of those functions existed. The agents are still pointing at the original stateless tool definitions. The ctx.store counter never increments because the tools the agents are actually calling never touch the context.

The course presents the state tracking result as if it works out of the box. It does not. The fix requires three explicit steps that the notebook collapses into one:

from llama_index.core.workflow import Context

# Step 1 — redefine tools with ctx
async def add(ctx: Context, a: int, b: int) -> int:
    """Add two numbers."""
    cur_state = await ctx.store.get("state")
    cur_state["num_fn_calls"] += 1
    await ctx.store.set("state", cur_state)
    return a + b

async def multiply(ctx: Context, a: int, b: int) -> int:
    """Multiply two numbers."""
    cur_state = await ctx.store.get("state")
    cur_state["num_fn_calls"] += 1
    await ctx.store.set("state", cur_state)
    return a * b

# Step 2 — recreate agents with the new tool definitions
multiply_agent = ReActAgent(
    name="multiply_agent",
    description="Is able to multiply two integers",
    system_prompt="A helpful assistant that can use a tool to multiply numbers.",
    tools=[multiply],
    llm=llm,
)

addition_agent = ReActAgent(
    name="add_agent",
    description="Is able to add two integers",
    system_prompt="A helpful assistant that can use a tool to add numbers.",
    tools=[add],
    llm=llm,
)

# Step 3 — build workflow and run
workflow = AgentWorkflow(
    agents=[multiply_agent, addition_agent],
    root_agent="multiply_agent",
    initial_state={"num_fn_calls": 0},
    state_prompt="Current state: {state}. User message: {msg}",
)

ctx = Context(workflow)
response = await workflow.run(user_msg="Can you add 5 and 3?", ctx=ctx)
state = await ctx.store.get("state")
print(state)  # {'num_fn_calls': 1}

The root cause is a sequencing assumption the notebook never makes explicit: agent objects must be instantiated after the tools they will use are defined in their final form. In Python, passing a function to a constructor copies the reference at that moment. Redefining the function later does not update the reference inside an already-constructed agent. The notebook redefines the tools but never rebuilds the agents, so the workflow runs with stale bindings throughout.

This is a subtle error precisely because it produces no exception. The workflow runs, the response comes back, and state["num_fn_calls"] returns zero. Nothing tells you that the counter failed to increment. You have to know to look for it.

Share

What LlamaIndex Gets Right and Where It Stops

The five-stage pipeline decomposition is the framework’s most durable engineering contribution. Separating loading, indexing, storing, querying, and evaluation into distinct abstractions makes each independently testable, observable, and improvable. When retrieval quality degrades after a data update, you can isolate whether the problem is in ingestion, in chunking, in embedding, or in query strategy without tracing through a monolithic pipeline.

The workflow system provides a clean middle ground between fully unconstrained agentic loops and fully deterministic pipelines. For applications where the orchestration structure is known at design time but the reasoning within each step is not, it is the right abstraction level.

What the framework does not provide: the code-first action representation that gives smolagents its compositional advantage for multi-step computation, built-in import sandboxing for safe code execution, and the minimal surface area that makes smolagents straightforward to debug. LlamaIndex is more opinionated and more verbose. For applications centred on retrieval over structured data, that verbosity buys you precision. For applications centred on dynamic computation and tool composition, it may be overhead.

The Conclusion Worth Taking Away

The lesson from Unit 2.1 was that the binding architectural constraint in agentic systems is the context window. Every design pattern in smolagents, multi-agent decomposition, concise sub-agent outputs, bounded memory scoping, exists to manage a fixed token budget across a dynamic reasoning process.

Unit 2.2 adds a second constraint: the boundary between what a model knows parametrically and what it needs to retrieve at inference time. LlamaIndex is designed specifically to manage that boundary. Its architecture treats the retrieval pipeline as a first-class engineering problem, not an add-on, and provides the tooling to build, inspect, evaluate, and improve each stage independently.

Taken together, the two units present a picture of agentic architecture that is less about which framework you choose and more about which constraints your application is primarily fighting. If the constraint is context window management and compositional computation, smolagents offers a leaner approach. If the constraint is reliable retrieval over private, dynamic, or heterogeneous data, LlamaIndex provides the more complete toolkit.

Both frameworks are responses to the same underlying problem: language models are powerful reasoners but unreliable knowledge stores. The architecture you choose should be determined by which half of that problem dominates your use case.

All corrected notebooks with documented fixes are available in our GitHub repository. Each correction includes a comment explaining the root cause and the reasoning behind the fix.

One practical note before getting into the substance: we ran every Jupyter notebook in this unit end to end. We found multiple errors, ranging from broken import paths to missing dependencies to provider configuration issues that produce misleading failure messages. We fixed all of them, documented the root cause of each fix, and published the corrected versions to our GitHub repository. I will point to the specific sections where these errors appear so you know where to expect friction.

The Architectural Decision That Defines the Framework

Every agent framework makes one foundational choice: how does the model express what it intends to do? The majority of frameworks, including the tool-calling interfaces from OpenAI and Anthropic, represent actions as JSON. The model outputs a structured object specifying a function name and a set of arguments. A runtime layer parses that object and dispatches the corresponding function call.

smolagents takes a different position. Its primary agent type, CodeAgent, instructs the model to write executable Python directly. The output is not a description of a computation. It is the computation itself, which the framework runs inside a sandboxed interpreter.

This distinction is not cosmetic. When a model writes a JSON tool call, each action is atomic. The model can invoke one function at a time and must chain calls across successive turns, carrying intermediate results through the context. When a model writes Python, it can loop over a list of queries in a single step, store intermediate values in variables, compose functions, and branch conditionally. The reduction in required steps for equivalent tasks is measurable, not theoretical.

The empirical case for this approach comes from the paper “Executable Code Actions Elicit Better LLM Agents,” which the course cites directly. The mechanism behind the performance advantage is straightforward: large language models are trained on vastly more Python than they are on agentic JSON schemas. Code is a natural representation for these models in a way that structured action schemas are not.

The cost is an expanded attack surface. Code execution is categorically more dangerous than JSON parsing. smolagents addresses this through an import allowlist. Any module not explicitly declared in additional_authorized_imports is blocked at runtime. This is a necessary constraint and not an optional configuration. Anyone building production systems with this framework needs to treat the allowlist as a security boundary, not a convenience setting.

The Mechanics of the Agent Loop

The CodeAgent is built on a base class called MultiStepAgent, which implements a structured variant of the ReAct framework. Each iteration of the loop performs four operations: it serializes the agent’s accumulated memory into a list of chat messages, sends those messages to the model, parses the model’s response for an executable code block, and runs that block. The result is logged back into memory as an ActionStep object.

The memory structure is explicit. The system prompt occupies a SystemPromptStep. The user’s original task is stored in a TaskStep. Every action and its result lives in an ActionStep. This separation makes the agent’s reasoning history inspectable without parsing a monolithic context string.

The constraint that the loop makes visible immediately is token consumption. Every step adds to the context. Tool outputs, especially from web retrieval, can be verbose. A multi-step agent working on a complex research task will accumulate thousands of tokens of intermediate results before producing a final answer. The performance degradation as context fills is not gradual. It tends to be abrupt once the context window is substantially occupied, because the model begins losing access to early instructions and prior reasoning. This is the central engineering constraint of the framework, and of agentic systems generally.

Share Data Akkadian

Tool Design: Where Docstrings Become Bearing

Tools in smolagents are defined either through the @tool decorator or by subclassing the Tool base class. Both approaches produce the same interface: a name, a natural language description, typed and described inputs, and a declared output type.

@tool
def catering_service_tool(query: str) -> str:
    """
    This tool returns the highest-rated catering service in Gotham City.
    Args:
        query: A search term for finding catering services.
    """

The description field is the operative part. The model selects tools and constructs arguments based on the descriptions it receives. A vague description produces inconsistent tool selection. A precise, accurate description produces reliable behaviour. This is not a stylistic preference. It is a functional requirement. In conventional software, docstrings are documentation for human readers. In an agentic system, they are specifications that the model executes against. The quality of your descriptions determines the quality of your agent’s behavior as directly as any other design decision.

The framework’s integration surface is broad. Tools can be loaded from the Hugging Face Hub, from Gradio Spaces using Tool.from_space(), from LangChain via Tool.from_langchain(), and from any MCP server through ToolCollection.from_mcp(). The MCP ecosystem in particular is worth attention. It is growing rapidly and the ability to compose smolagents workflows with external MCP servers without writing custom adapters reduces integration friction significantly.

Agentic RAG: What Actually Changes

Standard retrieval-augmented generation is a fixed pipeline. A query arrives, a retriever fetches relevant chunks, those chunks are injected into the prompt, and the model generates a response. The model has no influence over how retrieval is performed. It receives whatever the pipeline returns.

Agentic RAG restructures this by placing retrieval inside the tool loop. The model can formulate its own search queries rather than having the raw user input forwarded to the retriever. It can evaluate what it retrieved and decide whether to search again with a different query. It can route between a local knowledge base and live web search depending on what the question requires. These are not incremental improvements on the standard pipeline. They represent a qualitative change in how information is accessed.

The practical relevance is clearest when information sources are heterogeneous. A system that needs to pull from internal documentation, live web data, and structured databases cannot use a fixed retrieval pipeline without making routing decisions at design time. Those decisions are often wrong because the correct routing depends on the content and intent of individual queries, which vary at inference time. An agentic retriever can make those decisions dynamically. The cost is additional latency and token consumption per query. That cost is worth paying in proportion to how much query routing complexity your application actually has.

The course uses a BM25 retriever over an in-memory corpus. BM25 is a keyword-based ranking function with no embedding dependencies, which makes it operationally simpler and faster than dense retrieval methods. For many practical knowledge bases, especially those where queries use domain-specific terminology that appears explicitly in documents, it performs comparably to embedding-based retrieval. The choice of retriever matters less than the course examples might suggest. The agentic loop, providing the ability to reformulate and retry, compensates for retriever limitations more effectively than upgrading the retriever itself.

Where we found errors in the notebooks: The RAG notebook has two errors that will stop execution. The RecursiveCharacterTextSplitter import path changed in a recent release of langchain-text-splitters and the notebook uses the old path, producing a ModuleNotFoundError. Separately, BM25Retriever requires the rank_bm25 package, which is not listed in the notebook’s dependency cell. Neither error message points clearly to its fix. We corrected the import path, added the missing dependency, and published the corrected version to our GitHub.

Multi-Agent Systems: Context Windows as Architecture

The multi-agent section is the most technically substantive part of the unit.

A single agent working on a complex, multi-step task will fill its context window. When context fills, performance degrades and inference cost increases. The structural solution is decomposition: break the task across multiple agents, each maintaining its own bounded memory.

The course implements a manager-worker pattern. A manager agent receives the overall task, delegates subtasks to a specialized web search agent through the managed_agents interface, and aggregates results to produce the final output. The web agent does not carry the manager’s planning context. The manager does not carry the full content of every webpage the web agent fetched. Memory is scoped by role.

CodeAgent | deepseek-ai/DeepSeek-R1
├── Tools: [calculate_cargo_travel_time, final_answer]
└── Managed agents:
    └── web_agent | CodeAgent | Qwen2.5-Coder-32B-Instruct
        └── Tools: [web_search, visit_webpage, calculate_cargo_travel_time]

This decomposition does not eliminate the context problem. It redistributes it across agents with smaller, bounded contexts. The manager still aggregates results, and verbose sub-agent outputs will fill the manager’s context in turn. The operational discipline required is to prompt sub-agents to return concise, structured outputs. This is a design constraint that has to be enforced through prompting, not through the framework itself.

The course also introduces planning_interval, which causes the manager agent to pause every N steps and reason explicitly about its progress before continuing. This costs tokens. On complex tasks where errors compound across steps, it recovers more value than it costs. On simple tasks it is unnecessary overhead. The decision is empirical and depends on the specific task structure.

Where we found errors in the notebooks: The multi-agent notebook specifies model identifiers that are no longer available on Together AI’s serverless API, which is the provider the course routes inference through via the provider="together" argument in InferenceClientModel. The course was written against a point in time when the models it references were actively served through Together AI’s inference endpoints. None of them are available there today. The notebook contains no warning that model availability on third-party serverless providers changes over time, so when the call fails, nothing in the error message tells you that the models have been pulled from the provider’s catalog. It reads as a generic inference failure, which makes it easy to spend time debugging authentication and network configuration when the actual problem is that the endpoint no longer exists. We checked Together AI’s current list of supported models, identified available replacements that match the capability profile the course relies on for each agent role, updated the notebook accordingly, and ran the full multi-agent pipeline through to completion to verify the fixes hold. The corrected notebook with current model identifiers is in our GitHub repository.

Vision Agents: Capability Bounded by Cost

The vision section covers two capabilities. The first is providing images at initialization: images are passed alongside the task prompt and processed throughout the agent loop by a vision-language model. The second is dynamic image capture during browser automation: the agent drives a browser through Selenium and Helium, captures screenshots at each step, and stores them in ActionStep.observations_images so the model can reason about the current state of a page.

The first capability is straightforward and broadly applicable. Document understanding, visual data extraction, and identity verification from reference images are all real production use cases that this approach handles well.

The second capability is more operationally complex and more expensive. A multi-step browser agent that captures a screenshot at every step is consuming vision tokens at each iteration of the loop. Vision-language models with reliable visual reasoning capabilities are substantially more expensive per token than text-only models. The course is transparent about requiring a model like GPT-4o for reliable results. That transparency is appropriate. Anyone who attempts to run these examples with a smaller or cheaper VLM will find the results degrade significantly, and the degradation is not always obvious from the outputs alone.

What the Framework Gets Right and Where It Stops

smolagents is genuinely minimal. Approximately 1,000 lines of code constitute the core. When something fails, the stack trace is short and the failure is usually locatable. This is an underrated property in a framework. Debugging agent behavior is already difficult because agents are stochastic. A framework with shallow abstractions reduces one source of opacity.

The code-first action representation is well-supported by the evidence cited in the course. For tasks requiring composition, iteration, and variable handling across steps, CodeAgent outperforms ToolCallingAgent in a way that matters for production reliability.

The observability integration with OpenTelemetry and Langfuse provides structured traces, step-level logs, and run replay. This is necessary for any deployment that needs to be maintained and improved over time.

What the framework does not provide: persistent memory across sessions, workflow state management for long-running processes, and tool-level access control within a single agent. These are infrastructure problems. They do not belong in a framework of this scope, and the framework is right not to attempt them. The extension points are clean enough that teams can build what they need on top.

The Conclusion Worth Taking Away

Agent frameworks share the same set of primitives: a model, a tool interface, a memory structure, and an execution loop. smolagents implements these with less abstraction overhead than most alternatives. That is its primary virtue and the source of its educational value.

The deeper lesson from this unit is that the binding constraint in agentic systems is not the model, not the tools, and not the framework. It is the context window. Every architectural pattern the course introduces, multi-agent decomposition, planning intervals, concise subagent outputs, bounded memory scoping, is a response to a single hard constraint. The model has a fixed context window and every token spent on intermediate state is a token unavailable for the next step.

That constraint is not going away. Context windows are growing, but so is the complexity of the tasks practitioners want agents to tackle. Understanding the constraint and designing around it is more durable knowledge than familiarity with any specific framework.

All corrected notebooks with documented fixes are available in our GitHub repository. Each correction includes a comment explaining the original issue and the reasoning behind the fix, so the repository serves as a secondary reference alongside the course material itself.

There are approximately 1.8 billion Muslims on earth. A growing and significant number of them are investing. And a growing number of those investors have turned their attention to cryptocurrency, one of the most consequential and contentious asset classes of the past decade. The question of whether crypto is halal is no longer abstract. It is a live financial and theological conversation happening across kitchen tables in Karachi, investment forums in London, and WhatsApp groups in Jakarta.

The problem is not that Muslim investors lack enthusiasm. The problem is that the tools they are using to make these decisions are, in most cases, not built on a coherent methodological foundation. They are built on convenience.

This is not a trivial distinction. The difference between a tool that produces a halal verdict because it has a checklist and a tool that produces one because it has a methodology grounded in fiqh al muamalat and financial analysis is the difference between performing wudu and actually being in a state of tahara. The outcome might look the same from the outside. The substance is entirely different.

The Scriptural Imperative Behind Every Financial Decision

Before we discuss screening technology, we need to understand why Muslims screen at all. This is not optional religious behavior layered on top of otherwise secular finance. It is a direct obligation derived from the Quran.

Allah says in Surah Al-Baqarah (2:275):

الَّذِينَ يَأْكُلُونَ الرِّبَا لَا يَقُومُونَ إِلَّا كَمَا يَقُومُ الَّذِي يَتَخَبَّطُهُ الشَّيْطَانُ مِنَ الْمَسِّ

“Those who consume riba cannot stand except as one stands who is being beaten by Satan into insanity.”

And in the same surah (2:278-279), the command is direct and clear:

يَا أَيُّهَا الَّذِينَ آمَنُوا اتَّقُوا اللَّهَ وَذَرُوا مَا بَقِيَ مِنَ الرِّبَا إِن كُنتُم مُّؤْمِنِينَ

“O you who have believed, fear Allah and give up what remains of riba, if you should be believers.”

The prohibition is not merely against receiving interest. The scholars of fiqh al muamalat, from Ibn Qudama in the Hanbali tradition to Ibn Abidin in the Hanafi school, have consistently held that knowingly placing one’s wealth in a structure that generates riba makes one complicit in that riba. This is the financial and spiritual foundation on which every credible Islamic screening methodology must stand.

In crypto, this extends beyond obvious interest. It encompasses yield-generating protocols built on debt, governance tokens whose value derives primarily from speculative liquidity provision, stablecoins backed by interest-bearing instruments, and the broader ecosystem of DeFi structures that dress up ribawi mechanics in unfamiliar technical language.

The Problem With Most Halal Crypto Screeners in the Market

Let us be precise about what the existing landscape looks like, because Muslim investors deserve honesty here rather than comfortable vagueness. Platforms like CryptoUmmah have performed a service by publishing lists of screened coins covering hundreds of assets and applying criteria across legitimacy, project compliance, token structure, and staking mechanics. The effort is genuine and the documentation is more thorough than most. However, the platform itself acknowledges a critical limitation: with the lack of general scholarly consensus on crypto, their opinions may develop over time. That is intellectually honest, but it also means the Muslim investor is making decisions based on a living, shifting verdict without always knowing when the assessment changed or why.

The Islamic Finance Guru at islamicfinanceguru.com has done some of the most rigorous public-facing analysis of individual cryptocurrencies, applying genuine fiqh reasoning to coins like Bitcoin, AAVE, Uniswap, and MakerDAO. Their analysis of AAVE, for example, correctly identifies that borrowing cryptocurrency and returning more of the same cryptocurrency is straightforwardly riba, irrespective of how the protocol frames the mechanic. This is the kind of reasoning that separates analysis from a list. The limitation is coverage and scalability: a detailed scholarly article about one coin every few months is not a screener.

Zoya is arguably the most polished product in the halal investing space, applying the AAOIFI screening methodology with daily data refreshes across tens of thousands of stocks. But Zoya is fundamentally a stock screener. The AAOIFI financial ratios framework, which was designed for equity analysis of corporations with balance sheets, income statements, and debt structures, does not translate cleanly to a decentralized protocol or a utility token. Zoya does not solve the crypto problem.

Hudood takes an interesting approach by running assets through three independent screening criteria and presenting the majority verdict. This multi-standard approach acknowledges the genuine scholarly disagreement that exists. The limitation is that aggregating multiple methodologies without a unifying fiqhi framework risks producing a verdict that satisfies the average of different standards rather than the substance of any one of them.

HalalCryptoGuide and SarafScreening both provide useful starting points for investors, but both explicitly state that their classifications have not been verified by Islamic scholars or muftis, and that accuracy cannot be guaranteed. This is a responsible disclosure. It is also a significant caveat for anyone using these platforms to make real financial decisions.

What the HalalFinanx Crypto Screener Does Differently

The HalalFinanx crypto screener, available at halalfinanx.com, approaches the problem from a fundamentally different angle. Rather than attempting to be the single source of truth for every coin verdict, it functions as a curated and vetted directory that sits at the intersection of methodology, accessibility, and transparency.

The distinction matters enormously. When a platform tells you a coin is halal, the more important question is always: halal according to what? On what basis? Reviewed by whom? How recently? A directory that is transparent about the frameworks it applies and the scholars or methodologies it references is giving you something more valuable than a verdict. It is giving you the reasoning infrastructure to ask the right questions yourself.

The screener applies what can be understood as a multi-layered framework rooted in classical Islamic finance principles adapted for the specific nature of digital assets. This means the evaluation is not simply whether a project avoids obvious haram industries. It goes deeper, into the actual tokenomics, the revenue model of the underlying protocol, the nature of any yield-generating mechanisms, and critically, the distinction between different types of tokens: utility tokens, governance tokens, equity tokens, and stablecoins each raise distinct fiqhi questions that a single binary halal or haram verdict cannot adequately capture.

This is not a cosmetic difference. Consider stablecoins as an example. USDT is backed largely by US Treasury bills, which are interest-bearing government debt instruments. A screener that simply asks whether Tether is used for gambling misses the underlying problem entirely. A methodology that examines the backing assets and asks whether holding USDT constitutes indirect ownership of interest-bearing instruments is asking the fiqhally correct question. The Islamic Finance Guru has noted that while USDT is backed by interest-bearing instruments, the question of whether the holder is technically considered an owner of a portion of that debt is genuinely contested. HalalFinanx’s approach of surfacing this complexity rather than flattening it into a verdict is precisely what a serious Muslim investor needs.

The Fiqhi Framework Behind Serious Crypto Screening

Any credible methodology must navigate three foundational prohibitions from the classical tradition. First, riba, which in the context of crypto manifests most clearly in lending protocols, yield farming structures that operate like interest accounts, and any mechanism where the return is predetermined and guaranteed regardless of underlying economic activity.

Second, gharar, which refers to excessive uncertainty or ambiguity in a contract. The Quran’s broader instruction toward transparency in financial dealings, reflected in Surah Al-Baqarah (2:282) regarding the documentation of debts and contracts, establishes a principle that financial structures must be clear. Many DeFi protocols deliberately obscure their mechanics behind layers of smart contract complexity. This opacity itself constitutes a form of gharar that should concern any Sharia-conscious investor, regardless of whether the underlying economics are otherwise acceptable.

Third, maysir, gambling or speculation without underlying economic utility. The question of whether cryptocurrency investment in general constitutes maysir is one of the most debated issues in contemporary Islamic finance. The stronger scholarly position, held by institutions like the Accounting and Auditing Organization for Islamic Financial Institutions and scholars like Mufti Taqi Usmani, is that speculation is not inherently haram. What is prohibited is speculation that resembles gambling in structure: zero-sum outcomes, no underlying value creation, and participation driven purely by chance rather than analysis. Leverage trading in crypto, futures and derivatives markets, and meme coin speculation clearly fall on the haram side of this line. Spot ownership of a utility token with a genuine economic function does not clearly violate this principle, though uncertainty remains.

A screener that applies all three of these lenses systematically, and that distinguishes between them rather than collapsing them into a single verdict, is doing the work that the Muslim investor actually needs.

Practical Implications for the Muslim Crypto Investor

Understanding the methodology is not merely academic. It has direct practical implications for how you construct a portfolio and evaluate individual assets. Bitcoin, which the majority of credible scholars now treat as a permissible digital asset given its finite supply, transparent blockchain, and absence of an interest-generating structure in its base layer, is a reasonable starting point. But holding Bitcoin through a platform that automatically earns yield on your idle BTC through a lending mechanism transforms a halal holding into a potentially impermissible one. The asset itself does not change. The structure of your exposure does.

Ethereum presents more complexity. The transition to proof of stake introduced a staking mechanism whose Sharia compliance depends entirely on how you participate. Direct staking, where you are a validator contributing computational work to network security and receiving transaction fees as compensation for a genuine service, is structured differently from depositing ETH into a liquid staking protocol that operates more like a managed interest-generating account. A screener that tells you ETH is halal without distinguishing between these participation structures is not giving you the information you need.

Protocols like AAVE, Compound, and similar lending platforms are, in the view of most careful analysts applying classical fiqh reasoning, not permissible to use for yield generation. The mechanics are ribawi at their core, regardless of the technical novelty of the smart contract framework. This is not a close call.

For governance tokens whose value is inextricably linked to a protocol’s fee revenues, the analysis requires examining whether those revenues come from halal activity. Uniswap’s UNI token derives value from transaction fees on a decentralized exchange, which is a permissible activity if the exchanges being facilitated are themselves permissible spot transactions. This is different from MKR, whose value is tied to the MakerDAO lending mechanism, which operates on interest-based principles.

Why Methodology, Not Lists, Is the Standard You Should Hold Screeners To

The broader point this analysis leads to is one that Muslim investors should internalize as a matter of both financial and spiritual discipline. A halal screener that cannot explain its methodology is not a tool. It is an opinion dressed up as analysis.

The HalalFinanx crypto screener’s value proposition rests on transparency about how it evaluates assets, not simply on the volume of assets it covers or the accessibility of its interface. In a space where the same coin can appear as halal on one platform and haram on another, the investor who understands the reasoning behind each verdict is in a fundamentally stronger position than one who is simply choosing between verdicts.

Allah reminds us in Surah Al-Hashr (59:18):

يَا أَيُّهَا الَّذِينَ آمَنُوا اتَّقُوا اللَّهَ وَلْتَنظُرْ نَفْسٌ مَّا قَدَّمَتْ لِغَدٍ

“O you who have believed, fear Allah. And let every soul look to what it has put forth for tomorrow.”

The instruction to look at what we put forth is not passive. It is an active command to examine, to know, and to take responsibility for our choices. Using a screening tool without understanding its methodology is, from this perspective, a form of delegating that responsibility without actually fulfilling it.

Conclusion: The Intelligent Muslim Investor’s Approach

The cryptocurrency market is not going away, and Muslim investors who categorically avoid it on grounds of uncertainty alone are not making a religiously required decision. Uncertainty, in Islamic jurisprudence, triggers ijtihad, the scholarly exercise of informed judgment, not blanket prohibition.

What is required is that your investments be made through a process that is honest about what it knows, transparent about what it does not know, and grounded in the fiqhi principles that have governed Islamic commercial law for fourteen centuries.

HalalFinanx’s crypto screener is, in this sense, a tool built for the serious investor rather than the convenience-seeker. It does not simply tell you what is halal. It gives you the framework to understand why, and in doing so, it respects both your intelligence and your faith.

That is the standard every Muslim investor should demand. Not a list. A methodology.

This article is written for educational purposes. It does not constitute a fatwa or formal Sharia ruling, and Muslim investors are encouraged to consult qualified Islamic finance scholars before making investment decisions. All financial markets carry risk.

Table of Contents

1. Library Database Overview

1.1. Database Tables

  1.1.1. Books Table

  1.1.2. Readers Table

  1.1.3. Borrow Table

  1.1.4. Authors Table

  1.1.5. BookAuthors Table

  1.1.6. Categories Table

  1.1.7. BookCategories Table

2. SQL Code

3. Sample Data Insertion

4. Explanation of Relationships

5. Interview Questions

  5.1. Beginner …

Read more

Read more

Read more

This is my summary and notes about this great book called Market Wizards, if you want to become a trader, this is a must in your life.

Who is the author?

The author is Mr. Schwager a recognized industry expert in futures and hedge funds and the author of a number of widely acclaimed financial books. Previously, Mr. Schwager was a partner in the Fortune Group (2001–2010), a London-based hedge fund advisory firm. His prior experience also includes 22 years as Director of Futures research for some of Wall Street’s leading firms, most recently Prudential Securities.

Michael Marcus

1. The mental health of the trader is key for his success and, therefore catching huge profitable winning trades is essential. He emphasises that letting winners ride is as important as eliminating losses early. In his own words, “If you don’t stay with your winners, you are not going to be able to pay for the losers.”

2. In addition to not overtrading, he points out the need to set an exit point on every trade. He believes that protective stops are very necessary because they force this commitment on the trader. He also suggested closing positions to achieve mental peace when one is losing money.

3. Following the advice of other experienced traders almost results in disaster because it combines the worst qualities of both traders. because of this, he suggested following your intuition when you are trading.

4. Last but not least, despite being an aggressive trader, he firmly believes in being careful in choosing your trades. He recommends waiting for trades where all key elements align in one direction. By doing this you have more probability of success on each trade. He suggests that making numerous trades when conditions are only slightly favourable is more to do with entertainment than trading success.

Bruce Kovner

1. Let’s look at a real example. One recent Friday afternoon, the bonds saw a quick drop after a long stable period. From what I could see, this change was unexpected. I was confident in selling the bonds, thinking that if my decision was correct, the market wouldn’t go back to a certain level of previous stability. That was my limit. I was at ease with this decision because I knew I would exit the trade if that occurred.

2. The first piece of advice is every trader needs to understand risk management. second always under-trade. No matter what the size of your position is, cut it at least in half. My experience with novice traders is that they trade three to five times too big. They are taking 5 to 10 per cent risks on a trade when they should be taking 1 to 2 per cent risks.

3. The market is impersonal, doesn’t care if you make money or not. Every time a trader says “I wish,” or “I hope,” he is engaging in a destructive way of thinking because it takes attention away from the diagnostic process.

4. Kovner lists risk management as the key to successful trading; he always decides on an exit point before he puts on a trade. He also stresses the need for evaluating risk on a portfolio basis rather than viewing the risk of each trade independently. This is critical when one holds highly correlated positions since the overall portfolio risk is likely to be much greater than the trader realizes.

5. For him, risk management is the key to successful trading. He also believes in risk management at a portfolio level. This is important if one holds highly correlated positions, as this can lead to disaster because the portfolio’s risk is larger than the trader understands.

Richard Dennis

1. So you’re waiting for signs that the market is recovering before joining? It’s more likely that I’ll align myself with a trend and decide to sell off quicker than a trend-following system would, due to intuition.

2. Have you ever thought about starting a new trade against the current trend? I’ve done it before — meaning, I’ve started trades against the trend. But as a general guideline, I don’t think it’s a good idea.

3. How do you know when you’ve made a mistake in a major role? What signals you should step down? If you’re making a loss on a deal after a week or two, that’s a sign you’re wrong. Even if you’re just breaking even, but a lot of time has passed, it likely means you’re wrong as well.

4. If a market goes up when it should go up, I might buy earlier. If it goes down when it should go up, I’ll wait until the trend is better defined.

5. Trade small because that’s when you are as bad as you are ever going to be. Learn from your mistakes. Don’t be misled by the day-to-day fluctuations in your equity. Focus on whether what you are doing is right, not on the random nature of any single trade’s outcome.

6. Dennis believes that one of the worst mistakes a trader can make is to miss a major profit opportunity. According to his estimate, 95 per cent of his profits have come from only 5 per cent of his trades.

7. One particularly useful piece of advice offered by Dennis is that the times when you least want to think about trading — the losing periods — are precisely the times when you need to focus most on trading.

Paul Tudor Jones

1. He launched the Tudor Futures Fund in September 1984 with $1.5 million under management. At the end of October 1988, each $1,000 invested in this fund was worth $17,482, while the total amount of money he managed had grown to $330 million.

2. When you are trading big size, you have to get out when the market lets you out, not when you want to get out.

3. As a floor trader, I learned that if the old high was at 56.80, many buy stops might be at 56.85. If the market is trading 70 bids at 75 offered, everyone in the trading ring wants to buy the market, trigger those stops, and sell into the stops — a common practice. With Eli’s teachings, as an upstairs trader, I managed this situation by selling half of my position at 75. This way, I don’t need to worry about selling all at once when the stops are hit. I always sell half of my position before new highs or lows and the rest after that.

4. I learned that even though markets look their very best when they are setting new highs, that is often the best time to sell. He instilled in me the idea that, to some extent, to be a good trader, you have to be a contrarian.

5. I want to make sure that I never have a double-digit loss in any month.

6. Don’t ever average losers. Decrease your trading volume when you are trading poorly; increase your volume when you are trading well. Never trade in situations where you don’t have control. For example, I don’t risk significant amounts of money in front of key reports, since that is gambling, not trading.

7. Don’t be a hero. Don’t have an ego. Always question yourself and your ability. Don’t ever feel that you are very good. The second you do, you are dead.

8. The other part is that I have always been a swing trader, meaning that I believe the very best money is to be made at the market turns. Everyone says you get killed trying to pick tops and bottoms and you make all the money by catching the trends in the middle. Well, for twelve years, I have often been missing the meat in the middle, but I have caught a lot of bottoms and tops.

9. Trading gives you an incredibly intense feeling of what life is all about. Emotionally, you live on the extremes.

10. Don’t focus on making money; focus on protecting what you have.

Gary Bielfeldt

1. If you can just learn discipline by using a trend-following system, even temporarily, it will increase your odds of being successful as a trader.

2. What are the key factors you focus on in fundamentally evaluating the T-bond market? The economy is the single most important factor. Four other important elements are inflation expectations, the dollar, the trade balance, and the budget deficit.

3. The most important is discipline. I am sure everyone tells you that. Second, you have to have patience; if you have a good trade, you have to be able to stay with it. Third, you need courage to go into the market, and courage comes from adequate capitalization. Fourth, you must have a willingness to lose; that is also related to adequate capitalization. Fifth, you need a strong desire to win.

4. You don’t just play every hand and stay through every card, because if you do, you will have a much higher probability of losing. You should play the good hands, and drop out of the poor hands, forfeiting the ante. When more of the cards are on the table and you have a very strong hand — in other words, when you feel the percentages are skewed in your favour — you raise and play that hand to the hilt. If you apply the same principles of poker strategy to trading, it increases your odds of winning significantly. I have always tried to keep the concept of patience in mind by waiting for the right trade, just like you wait for the percentage hand in poker. If a trade doesn’t look right, you get out and take a small loss; it’s precisely equivalent to forfeiting the ante by dropping out of a poor hand in poker. On the other hand, when the percentages seem to be strongly in your favour, you should be aggressive and try to leverage the trade similar to the way you raise good hands in poker.

Ed Seykota

1. Yes, the winning traders have usually been winning at whatever field they are in for years.

2. One of the best ways to increase profits is to do goal setting and visualizations to align the conscious and subconscious with making profits. I have worked with several traders to examine their priorities and align their goals. I use a combination of hypnosis, breathing, pacing, visualization, gestalt, massage, and so forth.

3. The traders usually either: Get much more successful Or realize they didn’t want to be traders in the first place.

Larry Hite

1. From the inception of trading in April 1981 through mid-1988, Mint Investment Company registered an average annual compounded return of over 30 per cent. But it is their consistency that is most impressive: their annual returns have ranged from a worst of plus 13 per cent to a best of plus 60 per cent. Their largest loss in any six months was only 15 per cent and under 1 per cent in any twelve months (not just calendar years).

2. There is a very important message here: People don’t change. That is why this whole game works. In 1637, tulips in Holland traded for 5,500 florins and then crashed to 50, a 99 per cent loss. Well, you might say, “Trading was relatively new then; these people were primitive; capitalism was still in its infancy. Today we are much more sophisticated.” In 1929 a stock like Air Reduction which traded at a high of $233 and after the crash fell to $31, a decline of 87 percent. OK, you might say, “The Roaring ’20s were crazy times, but now things are surely different.” Move ahead to 1961 and you find a stock called Texas Instruments trading at $207. It eventually dropped to $49, a decline of 77 per cent. In the 1980s, all you have to do is look at silver prices, which in 1980 reached a peak of $50 and subsequently fell to $5, a 90 per cent decline. The point is that people are the same, and never change.

3. Larry, how can you trade the way you do; isn’t it boring?” I told him, “I don’t trade for excitement; I trade to win.” It may be very dull, but it is also very lucrative. When I get together with other traders and they start exchanging war stories about different trades, I have nothing to say. To me, all our trades are the same.

4. The truth is that, while you can’t quantify reward, you can quantify risk so the very first rule we live by at Mint is: Never risk more than 1 per cent of total equity on any trade.

5. There are four kinds of trades or bets: good bets, bad bets, winning bets, and losing bets. Most people think that a losing trade is a bad bet. That is wrong. You can lose money even on a good bet. If the odds on a bet are 50/50 and the payoff is $2 versus a $1 risk, that is a good bet even if you lose. The important point is that if you do enough of those trades or bets, eventually you have to come out ahead.

6. We diversify in two ways. First, we probably trade more markets worldwide than any other money manager. Second, we don’t just use a single best system. To provide balance, we use lots of different systems ranging from short-term to long-term. Some of these systems may not be that good by themselves, but we don’t care; that is not what they are there for.

7. Managing risk is tracking volatility. When the volatility of a market becomes so great that it adversely skews the expected return/risk ratio, we will stop trading that market.

8. Essentially, our approach has three lights in determining the acceptance of trading signals. When the light is green, we take all signals. When the light is yellow, we will liquidate an existing position on a signal, but we will not put on a new position. Finally, when the light is red, we liquidate existing positions automatically, and we do not take any new positions.

9. If the market makes a new high, we get back in. For example, if a long position is stopped out on a money management rule without a sell signal being generated, the system will still be in a long mode and no buy signal will be generated, no matter how high prices go. If, however, a sell signal were generated, the system would begin monitoring for a buy signal.

10. Larry, when you are on a motorcycle, never argue with a car. You will lose. The same lesson applies to trading: If you argue with the market, you will lose.

11. If you don’t bet, you can’t win. If you lose all your chips, you can’t bet.

12. Mint trades in an extraordinarily wide spectrum of markets (nearly sixty in all), encompassing exchanges in the U.S. and five foreign countries and diverse market groups including stock indexes, interest rates, currencies, raw industrial goods, and agricultural commodities.

13. Volatility is continually tracked in each market to generate signals to liquidate or temporarily suspend trading in those markets where the risk/reward ratio exceeds well-defined limits.

Michael Steinhardt

1. If you have made a mistake, deal with the mistake; don’t compound it.

2. Always ask yourself: Why does he want to sell? What does he know that I don’t? Finally, you have to be intellectually honest with yourself and others. In my judgment, all great traders are seekers of truth.

3. One trait I have noticed among several great traders is their willingness and ability to take on a particularly large position when they perceive a major trading opportunity.

William O’Neil

1. The “C” stands for current earnings per share. The best-performing stocks showed a 70 per cent average increase in earnings for the current quarter over the same quarter in the prior year before they began their major advance. I am continually amazed by how many individual investors, and even pension fund managers, buy common stocks with unchanged or lower current quarter earnings. There is no reason for a stock to go up if the current earnings are poor. If, as our research demonstrated, the best stocks had large profit increases before they advanced rapidly in price, why should anybody settle for mediocre earnings? So, our first basic rule in stock selection is that quarterly earnings per share should be up by at least 20 to 50 percent year to year.

2. The “A” in our formula stands for annual earnings per share. In our studies, the prior five-year average annual compounded earnings growth rate of outstanding performing stocks at their early emerging stage was 24 per cent. Ideally, each year’s earnings per share should show an increase over the prior year’s earnings.

3. The “N” in our formula stands for something new. The “new” can be a new product or service, a change in the industry, or new management.

4. The “S” in the formula stands for shares outstanding. Ninety-five per cent of the stocks that performed best in our studies had less than twenty-five million shares of capitalization during the period when they had their best performance.

5. The “L” in our formula stands for leader or laggard. The 500 best-performing stocks during the 1953–1985 period had an average relative strength of 87 before their major price increase began.

6. The “I” in the formula stands for institutional sponsorship. Institutional buyers are by far the largest source of demand for stocks. Leading stocks usually have institutional backing. However, although some institutional sponsorship is desired, excessive sponsorship is not, because it would be a source of large selling if anything went wrong with the company or the market in general. This is why the most widely owned institutional stocks can be poor performers. By the time a company’s performance is so obvious that almost all institutions own a stock, it is probably too late to buy.

7. The “M” in our formula stands for the market. Three out of four stocks will go in the same direction as a significant move in the market averages.

8. It is naive to hold stocks that are going down because they pay dividends. If you are getting a 4 per cent dividend and the stock goes down 25 per cent, your net yield is a 21 per cent loss.

9. A worse habit is to average down in your buying, rather than up. If you buy a stock at $40 and then buy more at $30 and average out your cost at $35, you are following up your losers and mistakes by putting good money after bad. This amateur strategy can produce serious losses and weigh you down with a few big losers.

10. In other words, they are willing to risk their hard-earned money on what someone else says, rather than knowing for sure what they are doing themselves.

11. The majority of unskilled investors stubbornly hold onto their losses when the losses are small and reasonable.

12. Individual investors worry too much about taxes and commissions. Your key objective should be first to make a net profit. Excessive worrying about taxes usually leads to unsound investments in the hope of achieving a tax shelter.

13. Some investors have trouble making decisions to buy or sell. In other words, they vacillate and can’t make up their minds.

David Ryan

1. Stocks selection: I look at the five-year earnings growth record and the last two-quarters of earnings relative to the previous year’s levels. The quarterly comparisons show you if there is any deceleration in the earnings growth rate. For example, a 30 per cent growth rate over the last five years may look very impressive, but if the last two-quarters earnings were only up 10 per cent and 15 per cent, it warns you that the strong growth period may be over. Of course, those factors — the five-year earnings growth record and the earnings during the past two quarters — are combined in our earnings per share (EPS) rank.

2. The single most important advice I can give anybody is: Learn from your mistakes. That is the only way to become a successful trader.

3. The maximum he will risk on any trade is a seven per cent price decline.

4. Maintaining a trader’s diary is an essential element of Ryan’s approach. Every time he buys a stock, Ryan annotates the chart with his reasons for buying the stock.

Marty Schwartz

1. I always check my charts and the moving averages prior to taking a position. Is the price above or below the moving average? That works better than any tool I have. I try not to go against the moving averages; it is self-destructive.

2. Before putting on a position always ask, “Do I really want to have this position?” After a successful period, take a day off as a reward. I’ve found it difficult to sustain excellent trading for more than two weeks at a time. I’ve had periods where I can be profitable for twelve days in a row, but eventually, I just get battle fatigue.

3. The most important thing is money management, money management, money management. Anybody who is successful will tell you the same thing.

4. Why do most traders lose money? Because they would rather lose money than admit they’re wrong. What is the ultimate rationalization of a trader in a losing position? “I’ll get out when I’m even.” Why is getting out even so important? Because it protects the ego. I became a winning trader when I was able to say, “To hell with my ego, making money is more important.”

5. Also, don’t increase your position size until you have doubled or tripled your capital. Most people make the mistake of increasing their bets as soon as they start making money. That is a quick way to get wiped out.

6. The key lesson is that each trader must find his or her own best approach.

7. The second element behind Schwartz’s transition to success was a change in attitude. As he describes it, he became successful when his desire to win took precedence over his desire to be right.

8. Risk control is an essential element of Schwartz’s trading style, as attested to by his incredibly low drawdowns. He achieves this risk control by always knowing his “uncle's point” on any trade. No doubt, his approach of sharply reducing position size after large losses, as well as extended winning streaks, contributes heavily to his success. The rationale for reducing position size after a destabilizing loss is apparent.

James Rogers

1. That is why I don’t think of myself as a trader. I think of myself as someone who waits for something to come along. I wait for a situation that is like the proverbial “shooting fish in a barrel.”

2. I was short stocks and short calls. I don’t buy options. Buying options is another fast way to the poorhouse. Someone did a study for the SEC and discovered that 90 per cent of all options expire as losses. Well, I figured out that if 90 per cent of all long option positions lose money, that meant that 90 per cent of all short option positions make money. If I want to use options to be bearish, I sell calls.

3. But 95 per cent of the time when you go against that kind of hysteria, you are going to make money.

4. I remember why I became even more bearish on the weekend before October 19. The week before, [Federal Reserve Chairman] Alan Greenspan announced that the balance of trade was getting much better and things were under control. Two days later, the balance of trade figures came out, and they were the worst in the history of the world. Right away I said, “This guy is either a fool or a liar. He doesn’t have any idea what is going on.” Then on the weekend before October 19, you had [Treasury Secretary] Baker telling the world we were going to stick it to the Germans by letting the dollar go because the Germans weren’t loosening monetary and fiscal policy as Baker had demanded. It looked like the trade wars of the 1930s all over again. **was in a panic mode and I was already short! I called Singapore that Sunday night to add to my shorts. [Singapore opens earlier than we do.] So all those guys who came in on Monday to sell had very, very good reasons to sell, and there were no buyers around. There were no buyers because there was no reason for people to buy. Even the buyers were scared and bearish that Monday.

5. Greenspan, Baker, the fact that money was tight, the steady worsening of the balance of trade, and you had a market that had spiked up to 2,700 six weeks earlier. If you check, you will see that, during 1987, while the S&P and the Dow were going up, the rest of the market was quietly eroding. In December 1986, I shorted the financial stocks, and throughout 1987,1 didn’t lose any money, even though the Dow and the S&P were going through the roof.

Mark Weinstein

1. Because I was inexperienced. I don’t believe anyone ever gets wiped out in the market because of bad luck; there is always some other reason for it. Either you were off when you did the trade, or you didn’t have the experience. There is always a mistake involved.

2. Weinstein employs his own custom-designed state-of-the-art computer systems to constantly monitor technical indicators designed to measure changes in market momentum. Rather than use the standard values for these indicators, Weinstein uses his values, which he frequently adjusts for changing market conditions.

3. It helps to understand that Weinstein usually plays for quick profits and covers his trades within hours or even minutes. Even on position trades, Weinstein will usually take some profits quickly to assure a net profitable outcome. He also trades markets in rotation, quickly shifting his profits from market to market, always seeking the profit potential with the lowest perceived risk. Finally, Weinstein enjoys the support of a floor network that often puts him on the right side of the bid-asked spread.

4. When I trade at home, I often watch the sparrows in my garden. When I feed them bread, they take just a little piece at a time and fly away. They keep on flying back and forth, taking small bits of bread. They may have to make a hundred stabs at a piece of bread to get what a pigeon gets at one time, but that is why a pigeon is a pigeon. You will never be able to shoot a sparrow, it is just too fast. That is the way I day trade. For example, there are times during the day when I am sure that the S&P is going up, but I don’t try to pick the bottom, and I am out before it tops. I just take the mid-range where the momentum is greatest. That, to me, is trading like a sparrow eats.

5. Always do your homework.

6. Don’t be arrogant. When you get arrogant, you forsake risk control. The best traders are the most humble.

7. Understand your limitations. Everyone has limitations — even the best traders.

8. Be your person. Think against the herd, as they must lose in time. Don’t trade until an opportunity presents itself. Knowing when to stay out of the markets is as important as knowing when to be in them.

9. Your strategy has to be flexible enough to change when the environment changes. The mistake most people make is they keep the same strategy all the time. They say, “Damn, the market didn’t behave the way I thought it would.” Why should it? Life and the markets just don’t work that way.

10. Weinstein’s most traumatic trading experience occurred when he let a material goal interfere with his trading. This is a common theme that has surfaced in other interviews. It invariably seems to be a mistake to translate the potential profit or loss of trade into material terms.

Brian Gelber

1. I instinctively trade smaller and sometimes I just take a break. It is a good habit to wipe the slate clean and start fresh.

Tom Baldwin

1. He turned a profit from the beginning. He was a millionaire before his first year was up and has never looked back. Although he declines to discuss the extent of his winnings, $30 million appears to be a conservative estimate.

2. They trade too much. They don’t pick their spots selectively enough. When they see the market moving, they want to be in on the action. So, they end up forcing the trade rather than waiting patiently. Patience is an important trait many people don’t have.

3. The best traders have no ego. To be a great trader, you have to have a big enough ego only in the sense that you have confidence in yourself. You cannot let your ego get in the way of a trade that is a loser; you have to swallow your pride and get out.

4. To him, money is only a means of keeping score. By contrast, most traders tend to think of gains and losses in terms of their monetary implications — a frame of mind that only gets in the way of making trading decisions. For example, assume you originally planned to risk up to $5,000 on a trade, and quickly find yourself down by $2,000. If, at this point, you start to think of the trade in terms of money (for example, “the extra $3,000 can pay for my vacation”), you might well liquidate the position, even though you still believed in the trade. It is one thing to get out because you no longer like the position, but it is quite another matter to liquidate on impulse simply because you have translated the risk into tangible terms.

5. Baldwin’s point is that by bearing the pain just a little longer, you may be able to find a more favourable circumstance for liquidating. Of course, this philosophy should be applied only by disciplined traders: those with the ability to maintain a risk control strategy.

Tony Saliba

1. I consider myself a matrix trader. I trade everything on the screen as it interrelates with everything else. My basic strategy, however, was buying butterflies [a long or short position at one strike price balanced by an opposite position in higher and lower strike options — for example, long one IBM 135 call, short two IBM 140 calls, and long one IBM 145 call] and offsetting that with an explosion position.

2. By buying butterflies, do you mean you were long the middle or the wings [that is, the higher and lower strike price options]? Long the wings. Your risk is limited, and if the market does not move widely, time decay works in your favour. [Barring a favourable price move or an increase in volatility, the value of an option erodes steadily over time. In a relatively flat market, the premium erosion in options with a strike price near the market price — “middle” in a butterfly spread — will be greater than that of options further removed from the strike price — “wings” in the spread.] Of course, I tried to buy the butterflies as cheaply as I could. If I chained enough of them together, my profit zone would be fairly wide. Then I would do an explosion position in a more distant month.

3. What do you mean by the term “explosion position”? That’s my term. An explosion position is an option position that has limited risk and open-ended potential, which will profit from a large price move or an increase in volatility. For example, a position consisting of long out-of-the-money calls and long out-of-the-money puts would be an explosion position.

4. In other words, the explosion position is your money bet in case of a big move, while your scalping is paying the bills, that is, the time decay cost of the explosion position. Exactly.

Dr. Van K. Tharp

1. The simple truth is that most people are risk-aversive in the realm of profits — they prefer a sure, smaller gain to a wise gamble for a larger gain — and risk-seeking in the realm of losses — they prefer an unwise gamble to a sure loss. As a result, most people tend to do the opposite of what is required for success. They cut their profits short and let their losses run. If you think of trading as a game and that a mistake is not following the rules of the game, then it becomes much easier to follow these two rules. You should review your rules at the beginning of the day and review your trading at the end of the day. If you followed your rules, even if you lost money, pat yourself on the back. If you didn’t follow your rules, then mentally rehearse what you did and give yourself more appropriate choices in the future.

2. The second major problem people have is dealing with stress. Stress takes two forms: worry and the biological fight/flight response. Our brains have a limited capacity for processing information. If your mind is preoccupied with worry, that worry takes up most of the decision space, and you don’t have enough capacity left to perform effectively.

3. The solution for dealing with stress is to work on the causes and to develop stress protectors. I would recommend that people with this problem go into a stress management program. Also, it is important to understand that many stressful events are such because of the way you perceive them. Change those perceptions and you will change the event itself. For example, winners typically differ from losers in their attitude about losses. Most people become anxious about losses, yet successful speculators have learned that an essential ingredient to winning is to make it OK to lose. Since most people in our culture are taught that only winning is acceptable, most investors must change their beliefs about losses to become successful.

4. Another major problem is that many people allow their emotions to control their trading. I worked with a floor trader whose father was fairly successful. His father was not a good model for him, however, in that he was an alcoholic. As a result, he developed a part to protect himself from being like his father. He could make about $75,000 per year trading, but if he made any more than that, this part kicked in to make sure that he would not become too successful. He was the one who made $650,000 in about two months after we had completed the parts negotiation.

5. Finally, the last major problem is making decisions. Although there are many facets to decision-making problems, what most people do is bring their normal method of making decisions to trading the markets. For example, think about what you go through to buy a new car. You have to think about the model, deal, service, cost, accessories, etc. And it probably takes you a week or more to evaluate those factors and make a decision. Most people bring that same method of making decisions to trading and it just doesn’t work.

6. The best method that I’ve found for dealing with long, ineffective decision-making problems is to short-circuit them through a process called anchoring. That process is a little too involved to explain here.

7. There are three primary factors involved in duplicating success — beliefs, mental states, and mental strategies. If you duplicate the way the best traders use those three factors for every aspect of the trading task, then you can duplicate their results.

8. Generally, I find that top traders believe: Money is NOT important. It is OK to lose in the markets. Trading is a game. Mental rehearsal is important for success. They’ve won the game before they start.

9. If you are committed, then not only are you certain that you are doing the right thing, but somehow events just seem to occur to help you. If you are committed to being a trader, then you probably have an understanding at some level of what I’m talking about. You probably even understand that those events that help you might be big losses.

10. Problems they don’t own consist of blaming the markets, blaming floor traders or locals, blaming insider trading, blaming their broker, or blaming their system for what goes wrong.

11. When people own their problems, they discover that their results usually stem from some sort of mental state. Common examples are:I’m too impatient with the markets. I get angry at the markets. I’m afraid at the wrong time. I’m too optimistic about what will happen.

12. The realization that you are responsible for the results you get is the key to successful investing. Winners know they are responsible for their results; losers think they are not.

13. Well, mental state manipulation is what most people call discipline. I teach people a very simple procedure that they can use right away. For example, suppose you are at your desk and you become aware that you are in a mental state that you would like to change. Get up out of the chair. Walk away about four feet and then look at how you looked in that chair. Notice your posture, your breathing, and your facial expressions. Then imagine how you would look if you had the sort of mental state you would like. When you can see that clearly, sit down in the chair again and assume the position that you just imagined. That exercise works for almost any situation as it involves several important principles — changing your body posture, seeing yourself from a more objective viewpoint, and imagining a more resourceful state.

The Trader-A

1. I came to realize that one of my great errors had been failing to exploit major price moves that I had correctly anticipated

2. You say you don’t have enough time each day to do your work on the markets. You are too busy with your new job and writing your book. Here, let me show you something.” He started citing assumptions regarding the sales of my book, royalties per copy sold, and the total hours I had spent writing the book. He then scribbled various calculations on a yellow pad. He arrived at a final figure of $18.50 per hour. “Here,” he said, “this is what you are making on your book.” The tone of his voice implied that I was crazy to jeopardize tens of thousands of dollars in my trading for such a paltry sum. (, the $18.50 estimate is probably wildly overinflated, but remember this was a dream.)

   Thanks for reading Data Akkadian! This post is public so feel free to share it.

   Share

I wanted to take a moment to express my gratitude to each and every one of you for taking the time to read my article. Your support and engagement mean too much for me, and I am truly grateful for the opportunity to share my thoughts and insights with such an amazing audience. Following me inspires me to keep pushing myself to create valuable and meaningful content. Whether you’re a long-time subscriber or a new reader, please know that your presence makes a difference, and I am deeply thankful for your support. I am committed to delivering high-quality articles that inform, inspire, and resonate with you. Your feedback and comments fuel my passion for writing and motivate me to constantly improve. Thank you for being part of this auidence. I am honored to have you as part of my community, and I look forward to sharing many more articles and ideas with you in the future.

Concurrency indicates that two tasks progress simultaneously. For example, we need to order a pizza, and you have to write a story in Medium.

The synchronous way is you call the pizza shop and wait on hold until the pizza guy answers your call. Then once answered, you give him your order choice and your address and wait for confirmation. And a…

Read more

Linear regression is a statistical method to model the relationship between a dependent variable and one or more independent variables. Its goal is to determine the best linear equation that predicts the dependent variable's value based on the independent variables. This technique is frequently employed in economics, finance, and the social sciences.

Understanding the Basics: What is Linear Regression and How Does It Work?

Linear regression is a fundamental concept in statistics and machine learning that is used to understand the relationship between two variables. In simple terms, it helps us predict one variable based on the values of another variable.

The basic idea behind linear regression is to find the best-fitting line that represents the relationship between the independent variable (X) and the dependent variable (Y). This line is represented by the equation Y = mX + b, where m is the slope of the line and b is the y-intercept.

To find this fitting line, linear regression uses a technique called least squares estimation. This method calculates the sum of squared differences between the actual values of Y and the predicted values based on X. The goal is to minimise this sum of squared differences to obtain a line that covers these data points.

Linear regression can be used for various purposes, such as predicting sales based on spending on ads, estimating house prices based on square meters, or analysing trends in hotels. It provides valuable insights into relationships between variables and helps make informed decisions based on data analysis.

In conclusion, this effective method helps us establish relationships between variables and predict future trends using historical data. Upon understanding its fundamental principles, this useful instrument can be utilised across various sectors, encompassing business analytics, finance, healthcare, and more.

Key Concepts Explained

Let’s break down some key concepts that might sound difficult for you but they are crucial for understanding this powerful tool.

The equation for this linear regression model can be represented as:

y= β0 + β1x + ε

Where:

• β0 is the intercept,

• β1​ is the coefficient of the predictor variable x,

• ε is the residual (the difference between the predicted value and the actual value of y).

So, the equation explicitly shows the intercept (β0​), the coefficient (β1​), and the residuals (ε).

For example, if we have:

• Intercept (β0​) = 7

• Coefficient (β1​) = 12

• Residual (ε) = 1

Our linear regression equation would be:

y = 7+ 12x + 1

Sure, let’s simplify the explanations:

1. Coefficients: Coefficients are like the helpers in a math problem. In linear regression, they’re just numbers that you multiply with your predictor variables to make predictions about the outcome. They’re like sidekicks helping your main hero (the outcome) along.

2. Intercept: Think of the intercept as the starting point on a journey. In linear regression, it’s the point where your prediction line crosses the y-axis when all your predictors are zero. It’s like the starting point on a treasure map, giving you a clue about where your predictions begin.

3. Residuals: Residuals are the leftovers from your predictions that don’t quite fit perfectly. They’re the difference between what happened and what your model predicted. Picture them as those last few pieces in a puzzle that just don’t fit, no matter how hard you try.

Evaluating Model Performance

Mean squared error (MSE) defines how big is the error in regression models. Its definition is the average squared difference between the observed and predicted. When MSE equals zero means model has no error. When there is error, its value increases.

Root mean square error or root mean square deviation is one of the most commonly used measures for evaluating the quality of predictions in regression models. It measures the distance between predictions and true values using Euclidean distance.

Mean absolute error (MAE) is a popular metric. The MAE score is measured as the average of the absolute error values. It is the difference between the measured value and true value. For example, if a scale states 97 kilos but you know your true weight is 93 pounds, then the scale has an absolute error of 97 kg — 93 kg= 4 kg.

Which one is the best to use as error measurement, this depends on your use case and for robustness to outliers, MAE is less sensitive to outliers compared to MSE and RMSE. In MSE and RMSE, errors are squared before they are averaged, which gives a disproportionately large weight to large errors (outliers). This can skew the overall error metric if your data has many outliers or is highly variable.

Practical Application: How to Implement Linear Regression Using Python or R Programming Language

import numpy as np

class LinearRegression:
    def __init__(self, learning_rate=0.01, n_iterations=1000):
        self.learning_rate = learning_rate
        self.n_iterations = n_iterations
        self.weights = None
        self.bias = None

    def fit(self, X, y):
        n_samples, n_features = X.shape

        self.weights = np.zeros(n_features)
        self.bias = 0

        for _ in range(self.n_iterations):
            # Predicted values
            y_pred = np.dot(X, self.weights) + self.bias

            # Compute gradients
            dw = (1/n_samples) * np.dot(X.T, (y_pred - y))
            db = (1/n_samples) * np.sum(y_pred - y)

            # Update weights and bias
            self.weights -= self.learning_rate * dw
            self.bias -= self.learning_rate * db

    def predict(self, X):
        return np.dot(X, self.weights) + self.bias

def mean_squared_error(y_true, y_pred):
    return np.mean((y_true - y_pred) ** 2)

def root_mean_squared_error(y_true, y_pred):
    return np.sqrt(mean_squared_error(y_true, y_pred))

def mean_absolute_error(y_true, y_pred):
    return np.mean(np.abs(y_true - y_pred))

if __name__ == "__main__":
    np.random.seed(0)
    X = 2 * np.random.rand(100, 1)  
    y = 3 * X + 4 + np.random.randn(100, 1)
    
    # Create and train the model
    model = LinearRegression()
    model.fit(X, y)

    # Predictions
    X_test = np.array([[0.5], [1.5]])
    predictions = model.predict(X_test)
    print("Predictions:", predictions)
    
    # Evaluation
    mse = mean_squared_error(y, model.predict(X))
    rmse = root_mean_squared_error(y, model.predict(X))
    mae = mean_absolute_error(y, model.predict(X))
    print("Mean Squared Error:", mse)
    print("Root Mean Squared Error:", rmse)
    print("Mean Absolute Error:", mae)

No, my dear reader, the thing is I like to trade and invest, and I like to do it in normal stocks and cryptocurrency, my idea is to take all these topics machine learning, software design, scalability, infrastructure, trading and investing as progenitor topics which need to interact between each other to build a new idea. Let’s jump and talk about investing in cryptocurrency, as you know crypto is volatile if you compare it with normal stocks and bonds, it’s not for the faint of the heart.

This is because Bitcoin the king of the cryptocurrency world have more or less 48 per cent of volatility per year at the time of writing this post, depending on which exchange you measure the volatility, so what’s volatility, what’s hedging and how we can invest in Bitcoin without fear. First, let’s talk about hedging and we need to mention who’s the first guy who used this technique and made a profit with it.

What is Hedging?

Suppose you’re a fan of Real Madrid and you decide to bet 99 Euros on them to win the next UEFA Champions League. The bet pays 999 Euros if you get it right. After couple of months later, Real Madrid made it to the final and is set to play against FC Barcelona. You feel that this game is going to be a difficult one and that the outcome is uncertain historically speaking. You don’t want to risk losing your entire 99 Euros, so you decide to hedge your bet by placing a new bet against Real Madrid, it’s feels hard to bet against your team, but you have to protect your money.

You bet 500 Euros that FC Barcelona will win the game. It pays an even 500 Euros if you get it right. Now, if Real Madrid wins, you end up with 598 Euros (the 1098 Euros minus the 500 Euros you lose on the new bet). If FC Barcelona wins, you end up with 500 Euros (from the new bet). With just the initial bet on Real Madrid, your options were either you make 0 Euros which means you lose everything or 1098 Euros, depending on the outcome of the game. But by using the second approach in which you mitigate your risk, your options are now either 500 Euros or 598 Euros.

No matter what happens, your expected win is 549 Euros which is 500 plus 598 divided by 2, assuming that the game is a toss-up. By hedging your bet, you’ve minimized the losses you’re taking and can better predict the amount of money you will end up with. This is a simple explanation in Feynam’s terms regarding hedging. After this beautiful explanation of the concept of hedging let’s move on and talk about the father of hedge funds.

The Big Daddy

Alfred Winslow Jones was described in the New York Magazine of 1968 essay as “big daddy”. Jones created two investment ideas that helped frame his portfolio. Jones developed equilibrium by going long in potential shares and short-selling in less promising shares, which means he borrowed and sold them, predicting that their value would decline. By being some stocks long and others short he protected his fund from market fluctuates.

After hedging out market risk in the way commented before, he felt secure in leveraging his bet with borrowed money. This formula of hedging and leverage had a miracle influence on Jones’s portfolio of stocks. The same strategy could be put in bonds, futures, options, swaps, and any combination of these financial instruments this was its true magic. We can say Jones built this formula by chance rather than by purpose, He pioneered a platform for complex strategies more than he could contemplate.

What is Bitcoin?

It’s just a list of names. Not only do their names appear, but so does their bank account amount. Not nearly the technological revolution you anticipated? Let’s take a moment to consider what makes this list so unique. But first, recognise that our current banking system is nothing more than a list of names and balances. Our society has already made the switch to digital currency. We know how much money we have depending on the balance in our bank account. Your bank does not truly have that exact amount of money in a vault somewhere; it is only a figure in its database.

What distinguishes this Bitcoin list? This list is available to everyone we know. Not only does the bank know how much money everyone has, but everyone who possesses a copy of the list does as well. Strange, isn’t it? Simply ask for a copy! Because everyone has access, the list is incredibly easy to discover. You’ll see below that Alice is willing to share her copy. Why is this shared list useful?

Let’s assume Bob and Dave go out to dinner one night, and Bob, as usual, forgets his wallet. Dave pays the bill. But now that we’re in the twenty-first century. Bob realises he has two options for repaying Dave. Dave owes Bob £19 for supper. Let’s look at how Bob would pay Dave.

Using a Bank: I’m sure you’ve heard of this approach. You tell your bank who you want to transfer money to and how much you want to send. The bank will rearrange the numbers in its database such that Bob now has £19 less and Dave has £19 more.

Using Bitcoin: There is no need for a bank while using Bitcoin. Remember, this is only a list, and Bob and Dave both have a copy. So Bob deducts £19 from the balance next to his name and adds £19 to the amount next to Dave’s name. Money was delivered without the use of an intermediary

What is Volatility?

Volatility in simple terms, is how fearful something is of going up or down. It’s a measurement of fear and uncertainty among investors. Sometimes prices of things (bonds, commodities, crypto, stocks) move big and quickly over a short period, and this movement can go down or up. You can measure volatility in both directions up or down.

The volatility of the crypto market is a measure of how much the entire value of the crypto market changes up and down. Individual cryptocurrencies, such as Bitcoin, might be deemed volatile in addition to the market as a whole. More specifically, volatility may be calculated by examining how much an asset’s price differs from its average price.

Standard deviation is a statistical measure of market volatility that measures how far prices diverge from the average price. If prices fluctuate inside a limited trading range, the standard deviation will be low, indicating little volatility. In contrast, if prices change rapidly up and down, the standard deviation returns a high result, indicating extreme volatility. You can see in the image below the higher volatility of Bitcoin in October 2021, with the volatility measured by the yellow line in the second portion of the graphic.

Hedging Techniques

In most cases, hedging tactics entail the use of financial products known as derivatives. Options and futures are two of the most used derivatives. You may use derivatives to create trading strategies in which a loss in one investment is compensated by a gain in another. First, we will talk about the futures contract, its definition, simple explanation, an example, and use cases of futures contracts. Second, we are going to dive into options, options are complex products and you need to read and learn about options as much as you can.

Futures

A futures contract is an agreement between two parties to buy or sell an asset at a certain time in the future for a certain price. futures contracts are normally traded on an exchange. A simple explanation of the future is the following. Let’s imagine you have a car, and you want to sell it to your friend, but you can’t meet your friend right now, because both of you are living in different cities. So, you both agree to make a special agreement. You decide to sell the car to your friend at a fixed price, but the actual exchange will happen later when you meet. That special deal you made is like a futures contract.

For example, say you own an ice cream shop and you want to make sure that you always know how much you’ll pay for things like milk. You could buy a futures contract of milk from the CME group, which means you’re saying that no matter how expensive the milk gets, you’ll pay the price you agreed to in the contract on the agreed-upon date. That way, you know how much you’ll spend and you can plan accordingly. Is like insurance against the spike in the price of milk so you can protect yourself from this event.

There are two main strategies for the future:

The first technique is hedging and here you have two scenarios:

1. When the price of the underlying asset goes up you need to take a short position against loss.

2. When the price of the underlying asset goes down you need to take a long position against loss.

The second technique is speculation and here you have two scenarios:

1. Long position makes money if the price of the underlying asset goes up, and loses money if the price of the underlying asset goes down

2. Short position makes money if the price of the underlying asset goes down, and loses money if the price of the underlying asset goes up

Forward contracts are the same as Futures Contract but operate in Over Counter (OTC).

Options

Let’s start by talking about options in simple terms. first, we will introduce the definition, then we move on to the subject-specific vocabulary, use cases and last but not least how they work in the real economy. In this simple explanation, we are buying a car as our example, so an option is a contract that gives you the right but not the obligation to buy or sell a car in six months from the car’s dealer with the price of 20K British pounds, and you only pay 200 pounds for the contract. Building upon this easy example we introduce the terms to make you understand options.

Exercise price or Strike: This is the price of the underlying asset (the car), the price of buying or selling.

Premium: This is the cost of the contract, the money that you have to pay in exchange for this contract, is the price of the option.

Expiration date: This is the date on which you’ll buy or sell the car from the dealer, in this case, is six months.

Underlying asset: The car

Type: you have two types here, it can be Put which is selling the car to the dealer, or it can Call which is buying the car from the dealer.

There are two main use cases of options:

1. The first use case is hedging your risk, imagine you have a cryptocurrency like Bitcoin and you want to protect this asset from losing, so you need to use the put option, if the asset decreases in value, the option increases in value.

2. The second use case is speculation, your betting on the market going up or down by trading options.

How do options work?

There are two types of options, calls and puts, and for each type of option there are two scenarios short and long or (sell and buy):

Buying(long) a call option, gives you the right but not the obligation to buy the underlying asset (Bitcoin in our case) at the strict price (exercise price).

Buying(long) a put option gives you the right but not the obligation to sell the underlying asset (Bitcoin in our case) at the strict price (exercise price).

Let’s see a simple example:

• Calls: A call is the option to buy 100 shares of a stock at a defined price (called a strike) by the expiration date. A buyer of the call thinks the price of the stock is going to go up, and the seller thinks that it will stay below the strike price.

• Buyer of the Call: Pays a premium to the writer of the call. This premium ranges wildly depending on how volatile the stock is (basically what are the odds that a stock will move up from here). A stock like Coke or Proctor and Gamble have very low premiums because there is very little risk they’ll run up more than a few dollars a share. The buyer of the call would generally only “exercise” (use) the option if the share price of the stock goes above their strike price plus the premium. The goal of someone who is buying a call is that a stock moves up in price (hopefully well past the strike price)

• Writer (Seller) of the Call: Sells the right to the buyer to sell them the stock at the strike price anytime before its expiration and immediately receives the premium in their account. Their goal is to have the stock stay below the strike price. If it does, generally the buyer of the call will not exercise the call. The seller keeps the premium. For many investors, they write what are called “covered calls,” meaning they own the stock that they would be obligated to sell. A “naked call” is when you do not own the stock, and would have to go out, purchase the stock, and then sell it (likely at a lower price than you purchased it) if the option was exercised.

  Thanks for reading Data Akkadian! This post is public so feel free to share it.

  Share

This my personal review about the book, Inside the Black Box: The Simple Truth About Quantitative Trading by Rishi K. Narang, goes into the field of quantitative trading. In this book, Narang gives readers an understanding of how quantitative trading functions, as well as its advantages and disadvantages. He also talks about the several methods quantitative traders employ to make money on the stock market.

Making financial judgments using algorithms and mathematical models is known as quantitative trading. Quantitative trading employs data and statistical analysis to find patterns and trends in the markets, compared to traditional trading, which depends on human perception and decision-making.

Quantitative traders employ software programs to conduct their trades and aim to make money by taking advantage of these patterns and trends.
In the first section of Inside the Black Box, Narang explains the fundamentals of quantitative trading, highlighting how it differs from conventional trading and the reasons it has grown in popularity recently. He also debunks a few popular myths and misconceptions regarding quantitative trading, such as the idea that it is a “black box” that is difficult to comprehend.

The significance of data in quantitative trading is one of the book’s key themes. Data, according to Narang, is the lifeblood of quantitative trading, and proficiency in data collection, cleaning, and analyzing is essential for success. He covers how to find lucrative trading opportunities using the many forms of data that quantitative traders employ, such as market data, news data, and alternative data.

Several chapters of Narang are devoted to analyzing the various trading approaches that quantitative traders employ. These include statistical arbitrage, mean reversion, trend following, and other techniques. He provides examples and thorough explanations of each tactic’s advantages and disadvantages.

He also demonstrates how to combine these tactics to construct more complicated trading systems. One of the most intriguing sections of Inside the Black Box is Narang’s discussion of risk management in quantitative trading. He emphasizes that while quantitative trading may be extremely rewarding, it also includes substantial dangers. Among these risks are model risk, execution risk, and market risk. Narang offers specific guidance on how to mitigate these risks, such as using stop-loss orders, control of position size, and diversifying using multiple instruments from different assets.

Overall, Inside the Black Box is a fantastic primer in the realm of quantitative trading. Narang writes in a simple, entertaining way that simplifies even the most complicated subjects. The book is packed with practical tips and examples that will benefit both new and seasoned traders. It’s also a great resource for investors who want to learn how quantitative trading works and how to utilize it to make money in the financial markets.

One of the book’s merits is its emphasis on the practical elements of quantitative trading. Narang isn’t afraid to talk about the risks and rewards of this form of trading. He is open about the limits of quantitative models and the hazards associated with their use. He also provides practical guidance on how to overcome these obstacles and manage these risks, making this book an invaluable resource for anybody interested in quantitative trading.

Another quality of the book is its thorough treatment of the subject. From the fundamentals of quantitative trading to the most recent research and breakthroughs in the area, Narang covers it all. He also includes a multitude of resources for further reading, including as research papers, books, and websites, making the book an excellent starting point for anybody interested in delving deeper into the area of quantitative trading.

The book’s possible flaw is that it might be quite technical at times. Narang’s use of mathematical language and vocabulary may be frightening to some readers. He does, however, a good job of describing these principles in layman’s terms and offering instances to demonstrate their practical uses. Readers who are inexperienced with quantitative methods will also benefit from this article.

Practical Concepts

Main

This module will contain the configuration of the providers that you need so that terraform can install them. Providers are plugins that terraform uses to connect with cloud providers, remote systems, and other APIs. When you run terraform init terraform providers are downloaded and installed as part of the terraform work…

Read more

How to Gather Requirements

This is the first step to create and build your data pipeline in any company, relax, believe in this process that I present to you right here, right now, and if you have a huge ego, try to control it, or leave outside your …

Read more

Initial Expectations vs. Reality

When I joined IndieVoice.app, I expected a community where creators, critics, and strategists could openly collaborate to build something meaningful. My background in website analysis and SWOT analysis positioned me to provide valuable insights, which I hoped would help IndieVoice.app refine its offerings. However, what began as a constructive relationship took a sour turn when my critical feedback was seemingly unwelcome, resulting in my removal from the platform. IndieVoice.app, which should value user insights, instead demonstrated a disturbing aversion to the truth, kicking out those who dared to voice constructive criticism.

User Experience Issues

One of the first signs of IndieVoice.app’s unprofessional approach is the poor user experience (UX) and user interface (UI) that greets users upon joining. The layout feels clunky and unintuitive, with essential features hidden behind layers of menus and options. Navigating through IndieVoice.app is an exercise in patience, with a confusing interface that seems more like a maze than a straightforward platform. This lack of polish indicates a lack of care and attention to detail in the overall user experience.

For a platform claiming to foster indie voices, having a seamless, user-friendly interface is essential. Instead, users are left to navigate a UI that feels half-baked. A strong platform supports its users by providing a reliable and intuitive experience. Unfortunately, IndieVoice.app seems to prioritize pushing users through its ecosystem rather than creating a comfortable, professional environment for them.

Professionalism and the Dismissal of Constructive Feedback

In my experience, a defining moment in evaluating the quality of a platform is how it handles feedback. After completing reviews for 11 websites and performing a SWOT analysis, I expected my critiques to be welcomed as a means for IndieVoice.app to improve. A thoughtful, well-intentioned critique can lead to substantial progress. Instead, my feedback was met not with an appreciation or even polite acknowledgement, but with an abrupt removal from the platform.

IndieVoice.app’s reaction to my honest feedback raises serious questions about its commitment to growth and improvement. In any professional setting, critical feedback is essential for refining a product. Dismissing someone who invests time and effort in providing constructive criticism suggests an unprofessional mindset, where only praise is valued. If IndieVoice.app cannot handle honest feedback, how can it expect to support and nurture the independent creators it claims to champion?

Integrity and Transparency: Values in Question

IndieVoice.app’s apparent unwillingness to accept critical feedback signals a deeper issue: a lack of transparency and commitment to user integrity. Dismissing individuals who bring forth ideas for improvement undermines the authenticity of the platform. For users, especially creators looking to build their brands, this kind of selective feedback system creates a misleading environment where only positive responses are tolerated.

Platforms dedicated to independent voices should encourage diversity of thought and critical input, not silence it. A platform that cannot engage in open dialogue with its users is unlikely to provide genuine support. Users may find themselves censored or alienated if they don’t align with IndieVoice.app’s narrow view of acceptable feedback.

The Strengths, Weaknesses, Opportunities, and Threats (SWOT) Analysis Revisited

The SWOT analysis I conducted for IndieVoice.app highlighted several areas for improvement, which, if addressed, could enhance the platform’s usability and relevance. Here’s a brief overview of what I found in each category:

• Strengths: IndieVoice.app has the potential to fill a niche for independent creators and niche audiences. The idea of giving smaller voices a platform is promising.

• Weaknesses: The platform’s UX and UI flaws hinder accessibility and make navigation frustrating. The aversion to constructive criticism limits the potential for growth, and the lack of transparency raises doubts about the platform’s professional integrity.

• Opportunities: IndieVoice.app could benefit greatly from adopting a more transparent approach and welcoming user feedback. Enhancing the platform’s design and functionality could also attract more serious users.

• Threats: Inflexibility and lack of receptivity to user feedback pose existential risks to the platform’s reputation. Competing platforms that embrace a more open, user-friendly, and constructive model could easily eclipse IndieVoice.app if it continues down this path.

Each of these points provides a roadmap for improvement, but unfortunately, IndieVoice.app seems resistant to evolving in ways that would better serve its community.

Competitive Alternatives

For those considering IndieVoice.app, there are numerous alternative platforms that embrace open dialogue and value user contributions. Sites like Patreon, Medium, or even Substack are known for their support of creator-driven content and their willingness to grow based on user feedback. These platforms foster user engagement, providing better UX/UI experiences and actively encouraging feedback to refine their services.

Creators looking for a platform that values both their voice and their opinions will likely find a more supportive environment elsewhere. Platforms that encourage diversity of thought, accept critical input and are committed to transparency offer a more stable foundation for anyone serious about their work.

Conclusion: Why Users Should Think Twice About IndieVoice.app

IndieVoice.app has the potential to become a valuable space for independent creators, but its current approach suggests a lack of professionalism and transparency. My experience reveals a platform that cannot handle constructive criticism and lacks respect for those who invest time in offering valuable feedback. Dismissing users for being honest is a troubling indicator that IndieVoice.app is more interested in preserving its image than improving its functionality.

For anyone considering using IndieVoice.app, it’s essential to remember that true collaboration is built on a foundation of respect, openness, and a commitment to improvement. A platform that disregards feedback and lacks transparency ultimately disrespects its users. IndieVoice.app’s potential remains unrealized due to its own reluctance to evolve. For now, creators looking for a genuine, professional environment to share their work would be better served to explore other platforms that value user insights and are committed to transparency and growth.

Let me tell you a story about scalability. Anybody can understand, so prepare your coffee or tea and enjoy the reading.

Consider a baker. To make the best artisan bread, you have to know exactly the right recipe of the bread and when to put it in the oven just at the right time and temperature. That’s a unique skill that not just anyone can …

Read more

In Technical Ter…

Read more