Chapter 11 — Complete Technical Implementation

Chapter 11 — Technical Implementation

Frontend/API architecture
Orchestrator service
Planner
Memory service
Retrieval service
Model router
Verification service
Event/queue system
Logging and monitoring

11.1 Introduction

This chapter converts the ACAI architecture from a conceptual design into a practical software engineering plan.

The objective is not to claim that the complete system already exists. Instead, this chapter specifies how a research team could build a working prototype using currently available software components.

A practical ACAI implementation should be modular:

                ┌───────────────┐
                │   Frontend    │
                └───────┬───────┘
                        │
                        ▼
                ┌───────────────┐
                │  API Gateway  │
                └───────┬───────┘
                        │
                        ▼
                ┌───────────────┐
                │ Orchestrator  │
                └───────┬───────┘
                        │
      ┌─────────────────┼─────────────────┐
      ▼                 ▼                 ▼
  Planner            Memory          Retrieval
      │                 │                 │
      └─────────────────┼─────────────────┘
                        ▼
                ┌───────────────┐
                │ Model Router  │
                └───────┬───────┘
                        │
              ┌─────────┼─────────┐
              ▼         ▼         ▼
            LLM-A     LLM-B     LLM-C
              └─────────┼─────────┘
                        ▼
                ┌───────────────┐
                │ Verification  │
                └───────┬───────┘
                        ▼
                ┌───────────────┐
                │   Response    │
                └───────────────┘

11.2 Recommended Prototype Architecture

A first implementation does not need a huge distributed infrastructure.

A practical prototype could use:

Frontend ↓ Backend API ↓ Orchestrator ├── Planner ├── Memory ├── Retrieval ├── Model Router └── Verification ↓ Foundation Model

Once this works, individual services can be separated.

11.3 Frontend

The frontend provides the interface through which users interact with ACAI.

A modern web implementation could contain:

/pages ├── Home ├── Chat ├── Projects ├── Memory ├── Documents └── Settings

The interface should expose only information users actually need.

For example, the user may see:

AI Assistant

[ Enter your request................ ]

[ Send ]

Status: Planning → Retrieving → Reasoning → Verifying

The internal reasoning implementation should not be confused with exposing private chain-of-thought. A production interface can instead show concise process status, such as "Searching documents" or "Verifying response," without revealing hidden reasoning traces.

11.4 API Gateway

The API Gateway receives requests from the frontend.

Example endpoints:

POST /api/chat POST /api/upload GET /api/session GET /api/memory POST /api/feedback GET /api/health

The gateway handles:

Authentication Request validation Rate limiting Session identification Routing Error handling 11.5 Orchestrator

The Orchestrator is the central controller.

Its job is to coordinate the other services.

Conceptually:

Request ↓ Intent ↓ Plan ↓ Memory ↓ Retrieval ↓ Model ↓ Verification ↓ Response

The Orchestrator should not contain the implementation details of every subsystem.

Instead:

Orchestrator │ ├── Planner Service ├── Memory Service ├── Retrieval Service ├── Model Service └── Verification Service

This keeps the architecture maintainable.

11.6 Planner Service

The Planner Service converts a large request into structured tasks.

Example:

Input: "Build an AI document analysis application."

Output:

Task 1: Requirement analysis

Task 2: Document ingestion

Task 3: Text extraction

Task 4: Embedding generation

Task 5: Vector search

Task 6: LLM integration

Task 7: Verification

Task 8: Testing

Each task can contain:

task_id description priority dependencies status required_capability 11.7 Task Graph

Instead of storing tasks only as a list, the planner can create a dependency graph.

      Requirement
           │
    ┌──────┴──────┐
    ▼             ▼
Document       Database
Processing         │
    │              │
    └──────┬───────┘
           ▼
      AI Pipeline
           │
           ▼
        Testing
           │
           ▼
      Deployment

This makes dependencies explicit.

11.8 Memory Service

The Memory Service provides persistent and temporary memory.

A practical implementation can separate:

Working Memory Short-Term Memory Long-Term Memory Semantic Memory Vector Memory

Example API:

POST /memory/store POST /memory/search GET /memory/{id} DELETE /memory/{id}

A memory record could conceptually contain:

memory_id user_id session_id content embedding importance created_at updated_at source 11.9 Memory Storage

Different memory types may use different storage technologies.

Example:

Relational Database │ ├── Users ├── Sessions └── Metadata

Vector Database │ └── Semantic Memories

Object Storage │ └── Documents / Files

Cache │ └── Temporary Session Data

There is no requirement that one database technology must handle every type of information.

11.10 Retrieval Service

The Retrieval Service handles external and internal knowledge.

Pipeline:

Query ↓ Query Expansion ↓ Search ↓ Filtering ↓ Ranking ↓ Context Selection

A hybrid system can combine:

Keyword Search + Semantic Search + Metadata Filtering

This can be more flexible than relying on only one retrieval method.

11.11 Document Processing

Uploaded documents first pass through an ingestion pipeline.

File Upload ↓ File Validation ↓ Text Extraction ↓ Chunking ↓ Metadata Extraction ↓ Embedding ↓ Vector Storage

For example, a PDF may be converted into text segments before being indexed.

Each segment should retain useful metadata such as:

document_id page_number section source timestamp chunk_id

This makes later citation and retrieval easier.

11.12 Context Builder

The Context Builder combines:

User Prompt + Relevant Memory + Retrieved Knowledge + Task Plan + Tool Results

into a structured model input.

Conceptually:

┌─────────────────────────┐ │ User Request │ ├─────────────────────────┤ │ Relevant Memory │ ├─────────────────────────┤ │ Retrieved Evidence │ ├─────────────────────────┤ │ Task Requirements │ ├─────────────────────────┤ │ Tool Results │ └─────────────────────────┘ │ ▼ Model Input 11.13 Model Router

The Model Router determines which model should process a task.

It can evaluate:

Task Type Model Capability Context Requirement Latency Target Cost Limit Availability

Example:

Programming ↓ Code-capable model

Long-context analysis ↓ Long-context model

Simple classification ↓ Small model

The router should also support a fallback model.

Primary Model ↓ Failure? ↓ Backup Model ↓ Continue 11.14 Tool Execution Layer

Some tasks require external tools.

Possible tools include:

Calculator Search Code Execution Database Query File Parser Image Analysis

The Tool Layer should use explicit permissions.

Model ↓ Tool Request ↓ Permission Check ↓ Tool Execution ↓ Validated Result ↓ Model

The model should not automatically receive unrestricted access to the host system.

11.15 Verification Service

The Verification Service receives the model's draft and performs quality checks.

Draft ↓ Claim Extraction ↓ Evidence Comparison ↓ Consistency Check ↓ Policy / Safety Check ↓ Final Review

Where applicable, verification should use independent evidence rather than simply asking the same generation process whether its own answer is correct.

11.16 Response Service

After verification, the Response Service prepares the user-facing output.

It handles:

Formatting Citations Markdown Error messages Structured outputs Streaming

Example:

Internal Result ↓ Response Formatter ↓ User Interface 11.17 Event-Driven Architecture

As the system grows, asynchronous events can reduce coupling.

Example:

Request Received ↓ Event Queue ↓ Planner ↓ Retrieval ↓ Model ↓ Verification

Possible events:

REQUEST_CREATED PLAN_CREATED MEMORY_RETRIEVED DOCUMENT_RETRIEVED MODEL_COMPLETED VERIFICATION_COMPLETED RESPONSE_CREATED

This approach can make distributed deployments easier to scale.

11.18 Caching

Caching can reduce repeated work.

Potential cache targets:

Embeddings Retrieval results Model metadata Frequently accessed documents Session information

Example:

Request ↓ Cache? ┌┴───────┐ Yes No │ │ ▼ ▼ Return Compute

Caching must account for freshness so that outdated information is not returned when current information is required.

11.19 Observability

A production implementation should provide three major observability capabilities:

Logs

What happened?

Metrics

How often and how quickly did it happen?

Traces

How did a single request travel through the system?

Example:

Request #123

API ↓ 20ms Planner ↓ 80ms Memory ↓ 15ms Retrieval ↓ 220ms LLM ↓ 1.8s Verification ↓ 300ms Response

This makes performance bottlenecks easier to identify.

11.20 Error Handling

Every service should return structured errors.

Example:

{ "error_code": "MODEL_UNAVAILABLE", "message": "Primary model unavailable", "retryable": true }

The Orchestrator can then decide whether to:

Retry ↓ Fallback ↓ Ask User ↓ Return Error

Blindly retrying every failure can increase cost and latency, so retry policies should have limits.

11.21 Prototype Deployment

A beginner-friendly prototype can initially run on one machine:

Frontend Backend Database Vector Store LLM

After the prototype becomes stable, components can be separated:

Frontend Server │ ▼ API Server │ ├── Planner ├── Memory ├── Retrieval ├── Router └── Verification │ ▼ Model Server

This staged approach avoids unnecessary infrastructure complexity during early research.

11.22 Implementation Sequence

A practical development sequence is:

Basic Chat API ↓
Foundation Model ↓
Planner ↓
Retrieval ↓
Memory ↓
Verification ↓
Model Router ↓
Tool Layer ↓
Evaluation ↓
Production Hardening

The important engineering principle is to make each stage work before adding the next major subsystem.

11.23 Minimum Technical Prototype

The first demonstrable ACAI prototype could therefore be:

            USER
              │
              ▼
           Web UI
              │
              ▼
           API
              │
              ▼
         Orchestrator
          /    |    \
         /     |     \
    Planner  Memory  Retrieval
         \     |     /
          \    |    /
              ▼
         Model Router
              │
              ▼
            LLM
              │
              ▼
         Verification
              │
              ▼
           Response

This is sufficiently small to build and sufficiently modular to expand into a larger research platform.

11.24 Chapter Summary

This chapter translated the ACAI concept into a practical software architecture. The proposed implementation uses an API Gateway, Orchestrator, Planner, Memory Service, Retrieval Service, Model Router, Tool Layer, Verification Service, and Response Service.

The most important implementation principle is incremental construction. A small working prototype should be created first, measured against a baseline, and then expanded component by component.

That approach makes it possible to determine which ACAI features actually provide measurable value rather than assuming that architectural complexity automatically produces better AI performance.

End of Chapter 11

Stay tuned for Chapter: 12 Complete End-to-End System Architecture.

🚀 Connect with Black Shadow Team Across the Web! 🌐

We are actively sharing our latest cybersecurity research, AI safety insights, ethical hacking content, and tech updates across multiple platforms. Follow and subscribe to stay updated with our official channels:

📝 Articles & Research Papers:

Medium: https://medium.com/@blackshadowteam.net

Substack: https://blackshadowteam.substack.com

Dev.to: https://dev.to/black\_shadow\_team

HackerNoon: https://hackernoon.com/u/black-shadow-team

Hashnode: https://hashnode.com/@black-shadow-team

Blogspot: https://black-shadow-team.blogspot.com/

💻 Code & Open Source:

GitHub: https://github.com/blackshadowteamnet-netizen

WordPress: https://profiles.wordpress.org/blackshadowteam

📱 Social Media & Updates:

X (Twitter): https://x.com/BlackShadoTeam