flare-agent Bài 11: Workspace — Filesystem & Document Processing trên R2

Series: Build Your Own AI Agent Framework trên Cloudflare Bài: 11 / 11 — @flare-agent/workspace

---

Vấn đề

Agent xử lý documents cần:

• Lưu file upload của user
• Đọc lại file để parse/extract
• Index content để search sau
• Chia sẻ files giữa các agents trong cùng session

Cloudflare Workers không có filesystem — nhưng R2 object storage có thể giả lập đủ tốt cho document usecase.

---

Thiết kế: R2 như filesystem

R2 bucket layout:
  workspaces/
    {workspaceId}/
      files/
        {filename}         ← raw files
      index/
        {filename}.meta    ← metadata JSON
      skills/
        {skillName}.md     ← reusable instructions

Mỗi workspace được isolated theo workspaceId — thường là userId hoặc sessionId.

---

Cấu trúc package

packages/workspace/
  src/
    types.ts              # FileEntry, WorkspaceConfig
    FileSystem.ts         # CRUD operations trên R2
    DocumentParser.ts     # PDF, Markdown, CSV parser
    SearchIndex.ts        # BM25 + Vectorize search
    SkillRegistry.ts      # Reusable agent instructions
    WorkspaceTools.ts     # Tool definitions cho agent
    index.ts

---

Types

// src/types.ts

export interface FileEntry {
  path: string;           // relative path trong workspace
  size: number;
  contentType: string;
  createdAt: number;
  metadata?: Record<string, unknown>;
}

export interface WorkspaceConfig {
  workspaceId: string;
  r2: R2Bucket;
  vectorize?: VectorizeIndex;
  ai?: Ai;                // cho embedding + extraction
}

export interface ParsedDocument {
  path: string;
  contentType: string;
  text: string;           // extracted plain text
  chunks: string[];       // split thành chunks cho RAG
  metadata: {
    title?: string;
    pages?: number;
    wordCount: number;
  };
}

export interface SearchResult {
  path: string;
  score: number;
  excerpt: string;        // đoạn text liên quan nhất
  metadata?: Record<string, unknown>;
}

---

FileSystem — CRUD trên R2

// src/FileSystem.ts
import type { FileEntry, WorkspaceConfig } from './types';

export class FileSystem {
  private prefix: string;

  constructor(private config: WorkspaceConfig) {
    this.prefix = `workspaces/${config.workspaceId}/files`;
  }

  private key(path: string) {
    // Sanitize path — tránh path traversal
    const clean = path.replace(/\.\.\/|\.\.\\/, '').replace(/^[\/\\]/, '');
    return `${this.prefix}/${clean}`;
  }

  // Write file
  async write(
    path: string,
    content: ArrayBuffer | string,
    contentType = 'text/plain'
  ): Promise<FileEntry> {
    const key = this.key(path);
    const body = typeof content === 'string'
      ? new TextEncoder().encode(content)
      : content;

    await this.config.r2.put(key, body, {
      httpMetadata: { contentType },
      customMetadata: {
        createdAt: Date.now().toString(),
        path,
      },
    });

    return {
      path,
      size: body.byteLength,
      contentType,
      createdAt: Date.now(),
    };
  }

  // Read file
  async read(path: string): Promise<ArrayBuffer | null> {
    const obj = await this.config.r2.get(this.key(path));
    if (!obj) return null;
    return obj.arrayBuffer();
  }

  // Read as text
  async readText(path: string): Promise<string | null> {
    const buf = await this.read(path);
    if (!buf) return null;
    return new TextDecoder().decode(buf);
  }

  // List files
  async list(prefix?: string): Promise<FileEntry[]> {
    const listPrefix = prefix
      ? `${this.prefix}/${prefix}`
      : this.prefix;

    const result = await this.config.r2.list({ prefix: listPrefix });

    return result.objects.map((obj) => ({
      path: obj.key.replace(`${this.prefix}/`, ''),
      size: obj.size,
      contentType: obj.httpMetadata?.contentType ?? 'application/octet-stream',
      createdAt: parseInt(obj.customMetadata?.createdAt ?? '0'),
    }));
  }

  // Delete
  async delete(path: string): Promise<void> {
    await this.config.r2.delete(this.key(path));
  }

  // Copy
  async copy(fromPath: string, toPath: string): Promise<void> {
    const content = await this.read(fromPath);
    if (!content) throw new Error(`File not found: ${fromPath}`);
    const obj = await this.config.r2.get(this.key(fromPath));
    await this.write(toPath, content, obj?.httpMetadata?.contentType);
  }

  // Move
  async move(fromPath: string, toPath: string): Promise<void> {
    await this.copy(fromPath, toPath);
    await this.delete(fromPath);
  }

  // Grep — tìm text trong files
  async grep(query: string, filePattern?: string): Promise<Array<{
    path: string;
    line: string;
    lineNumber: number;
  }>> {
    const files = await this.list(filePattern);
    const results = [];

    for (const file of files) {
      // Chỉ grep text files
      if (!file.contentType.startsWith('text/')) continue;

      const text = await this.readText(file.path);
      if (!text) continue;

      const lines = text.split('\n');
      lines.forEach((line, i) => {
        if (line.toLowerCase().includes(query.toLowerCase())) {
          results.push({ path: file.path, line: line.trim(), lineNumber: i + 1 });
        }
      });
    }

    return results;
  }
}

---

Document Parser

// src/DocumentParser.ts
import type { ParsedDocument } from './types';

const CHUNK_SIZE = 512;  // words per chunk
const CHUNK_OVERLAP = 50;

export class DocumentParser {
  async parse(
    path: string,
    content: ArrayBuffer,
    contentType: string
  ): Promise<ParsedDocument> {
    let text = '';
    let metadata: ParsedDocument['metadata'] = { wordCount: 0 };

    if (contentType === 'text/markdown' || contentType === 'text/plain') {
      text = new TextDecoder().decode(content);
    } else if (contentType === 'text/csv') {
      text = this.parseCSV(new TextDecoder().decode(content));
    } else if (contentType === 'application/pdf') {
      // Workers AI vision model extract text từ PDF
      throw new Error('PDF parsing cần Workers AI — xem phần bên dưới');
    }

    const wordCount = text.split(/\s+/).filter(Boolean).length;
    const chunks = this.chunkText(text);

    return {
      path,
      contentType,
      text,
      chunks,
      metadata: { ...metadata, wordCount },
    };
  }

  private parseCSV(csv: string): string {
    // Convert CSV thành readable text cho LLM
    const lines = csv.trim().split('\n');
    if (!lines.length) return '';

    const headers = lines[0].split(',').map((h) => h.trim());
    const rows = lines.slice(1).map((line) => {
      const values = line.split(',');
      return headers
        .map((h, i) => `${h}: ${values[i]?.trim() ?? ''}`)
        .join(', ');
    });

    return `Table with columns: ${headers.join(', ')}\n\n` + rows.join('\n');
  }

  private chunkText(text: string): string[] {
    const words = text.split(/\s+/);
    const chunks: string[] = [];

    for (let i = 0; i < words.length; i += CHUNK_SIZE - CHUNK_OVERLAP) {
      const chunk = words.slice(i, i + CHUNK_SIZE).join(' ');
      if (chunk.trim()) chunks.push(chunk);
    }

    return chunks;
  }
}

---

Search Index — BM25 + Vector

// src/SearchIndex.ts
import type { WorkspaceConfig, SearchResult, ParsedDocument } from './types';

export class SearchIndex {
  // Index key trong R2
  private indexKey: string;

  constructor(private config: WorkspaceConfig) {
    this.indexKey = `workspaces/${config.workspaceId}/index`;
  }

  // Index document sau khi parse
  async indexDocument(doc: ParsedDocument): Promise<void> {
    // 1. Lưu metadata vào R2
    await this.config.r2.put(
      `${this.indexKey}/${doc.path}.meta`,
      JSON.stringify({
        path: doc.path,
        contentType: doc.contentType,
        wordCount: doc.metadata.wordCount,
        preview: doc.text.slice(0, 200),
      })
    );

    // 2. Vector index nếu có Vectorize
    if (this.config.vectorize && this.config.ai) {
      await this.vectorIndex(doc);
    }
  }

  private async vectorIndex(doc: ParsedDocument): Promise<void> {
    const vectors = [];

    for (let i = 0; i < doc.chunks.length; i++) {
      const chunk = doc.chunks[i];

      // Tạo embedding
      const response = await this.config.ai!.run(
        '@cf/baai/bge-base-en-v1.5',
        { text: [chunk] }
      ) as any;

      vectors.push({
        id: `${doc.path}::chunk::${i}`,
        values: response.data[0],
        metadata: {
          path: doc.path,
          chunkIndex: i,
          text: chunk,
        },
      });
    }

    await this.config.vectorize!.insert(vectors);
  }

  // Semantic search
  async search(query: string, topK = 5): Promise<SearchResult[]> {
    if (!this.config.vectorize || !this.config.ai) {
      return this.keywordSearch(query, topK);
    }

    // Embed query
    const response = await this.config.ai.run(
      '@cf/baai/bge-base-en-v1.5',
      { text: [query] }
    ) as any;

    const results = await this.config.vectorize.query(
      response.data[0],
      { topK, returnMetadata: true }
    );

    return results.matches.map((m) => ({
      path: m.metadata?.path as string,
      score: m.score,
      excerpt: m.metadata?.text as string,
    }));
  }

  // BM25-style keyword search (fallback)
  private async keywordSearch(
    query: string,
    topK: number
  ): Promise<SearchResult[]> {
    const list = await this.config.r2.list({
      prefix: `${this.indexKey}/`,
    });

    const results: SearchResult[] = [];
    const terms = query.toLowerCase().split(/\s+/);

    for (const obj of list.objects) {
      const raw = await this.config.r2.get(obj.key);
      if (!raw) continue;
      const meta = JSON.parse(await raw.text());

      // Simple TF score
      const text = (meta.preview ?? '').toLowerCase();
      const score = terms.filter((t) => text.includes(t)).length / terms.length;

      if (score > 0) {
        results.push({
          path: meta.path,
          score,
          excerpt: meta.preview,
        });
      }
    }

    return results
      .sort((a, b) => b.score - a.score)
      .slice(0, topK);
  }
}

---

Skill Registry — Reusable Instructions

// src/SkillRegistry.ts
// Skills là markdown files hướng dẫn agent làm task cụ thể

export class SkillRegistry {
  private prefix: string;

  constructor(
    private r2: R2Bucket,
    private workspaceId: string
  ) {
    this.prefix = `workspaces/${workspaceId}/skills`;
  }

  async save(name: string, instructions: string): Promise<void> {
    await this.r2.put(
      `${this.prefix}/${name}.md`,
      instructions
    );
  }

  async get(name: string): Promise<string | null> {
    const obj = await this.r2.get(`${this.prefix}/${name}.md`);
    return obj ? obj.text() : null;
  }

  async list(): Promise<string[]> {
    const result = await this.r2.list({ prefix: `${this.prefix}/` });
    return result.objects.map((o) =>
      o.key.replace(`${this.prefix}/`, '').replace('.md', '')
    );
  }

  // Load skill thành system prompt addition
  async loadAsPrompt(name: string): Promise<string> {
    const instructions = await this.get(name);
    if (!instructions) throw new Error(`Skill "${name}" not found`);
    return `\n\n## Skill: ${name}\n${instructions}`;
  }
}

---

Workspace Tools — cho agent dùng

// src/WorkspaceTools.ts
import { tool } from '@flare-agent/core';
import type { WorkspaceConfig } from './types';
import { FileSystem } from './FileSystem';
import { DocumentParser } from './DocumentParser';
import { SearchIndex } from './SearchIndex';

export function createWorkspaceTools(config: WorkspaceConfig) {
  const fs = new FileSystem(config);
  const parser = new DocumentParser();
  const index = new SearchIndex(config);

  return [
    tool({
      schema: {
        name: 'read_file',
        description: 'Đọc nội dung file trong workspace',
        parameters: {
          type: 'object',
          properties: {
            path: { type: 'string', description: 'Đường dẫn file' },
          },
          required: ['path'],
        },
      },
      execute: async ({ path }) => {
        const text = await fs.readText(path);
        if (!text) return { error: `File not found: ${path}` };
        // Giới hạn 2000 chars để không overflow context
        return { content: text.slice(0, 2000), truncated: text.length > 2000 };
      },
    }),

    tool({
      schema: {
        name: 'write_file',
        description: 'Ghi nội dung vào file trong workspace',
        parameters: {
          type: 'object',
          properties: {
            path: { type: 'string' },
            content: { type: 'string' },
          },
          required: ['path', 'content'],
        },
      },
      execute: async ({ path, content }) => {
        await fs.write(path, content);
        return { success: true, path };
      },
    }),

    tool({
      schema: {
        name: 'list_files',
        description: 'Liệt kê files trong workspace',
        parameters: {
          type: 'object',
          properties: {
            prefix: { type: 'string', description: 'Lọc theo prefix' },
          },
        },
      },
      execute: async ({ prefix }) => {
        const files = await fs.list(prefix);
        return { files: files.map((f) => ({ path: f.path, size: f.size })) };
      },
    }),

    tool({
      schema: {
        name: 'search_documents',
        description: 'Tìm kiếm nội dung trong tài liệu đã index',
        parameters: {
          type: 'object',
          properties: {
            query: { type: 'string', description: 'Query tìm kiếm' },
            topK: { type: 'number', default: 5 },
          },
          required: ['query'],
        },
      },
      execute: async ({ query, topK }) => {
        const results = await index.search(query, topK ?? 5);
        return { results };
      },
    }),

    tool({
      schema: {
        name: 'parse_and_index',
        description: 'Parse và index document để có thể search sau',
        parameters: {
          type: 'object',
          properties: {
            path: { type: 'string', description: 'Path file cần index' },
          },
          required: ['path'],
        },
      },
      execute: async ({ path }) => {
        const buf = await fs.read(path);
        if (!buf) return { error: `File not found: ${path}` };

        const files = await fs.list();
        const file = files.find((f) => f.path === path);
        const contentType = file?.contentType ?? 'text/plain';

        const doc = await parser.parse(path, buf, contentType);
        await index.indexDocument(doc);

        return {
          success: true,
          path,
          wordCount: doc.metadata.wordCount,
          chunks: doc.chunks.length,
        };
      },
    }),

    tool({
      schema: {
        name: 'grep',
        description: 'Tìm kiếm text pattern trong files',
        parameters: {
          type: 'object',
          properties: {
            query: { type: 'string' },
            filePattern: { type: 'string', description: 'Lọc theo prefix path' },
          },
          required: ['query'],
        },
      },
      execute: async ({ query, filePattern }) => {
        const results = await fs.grep(query, filePattern);
        return { results: results.slice(0, 20) }; // max 20 kết quả
      },
    }),
  ];
}

---

Dùng trong Worker

// apps/worker/src/agents/document.ts
import { Agent } from '@flare-agent/core';
import { createWorkspaceTools, SkillRegistry } from '@flare-agent/workspace';

export function createDocumentAgent(env: Env, workspaceId: string) {
  const workspaceConfig = {
    workspaceId,
    r2: env.R2,
    vectorize: env.VECTORIZE,
    ai: env.AI,
  };

  const skills = new SkillRegistry(env.R2, workspaceId);

  return new Agent({
    name: 'document-agent',
    model: { provider: 'groq', model: 'llama-3.3-70b-versatile' },
    memory: 'kv',
    systemPrompt: async (ctx) => {
      // Load skill từ workspace nếu có
      const skillList = await skills.list();
      const skillPrompts = await Promise.all(
        skillList.map((s) => skills.loadAsPrompt(s))
      );

      return [
        'Bạn là agent xử lý tài liệu.',
        'Có thể đọc, tìm kiếm và phân tích files trong workspace.',
        ...skillPrompts,
      ].join('\n');
    },
  }).use(...createWorkspaceTools(workspaceConfig));
}

// Route upload file
app.post('/workspace/upload', async (c) => {
  const formData = await c.req.formData();
  const file = formData.get('file') as File;
  const userId = c.req.header('X-User-Id') ?? 'anon';

  const fs = new FileSystem({
    workspaceId: userId,
    r2: c.env.R2,
  });

  const buf = await file.arrayBuffer();
  const entry = await fs.write(file.name, buf, file.type);

  return c.json({ success: true, file: entry });
});

// Route chat với document agent
app.post('/workspace/chat', async (c) => {
  const { input, sessionId, userId } = await c.req.json();

  const agent = createDocumentAgent(c.env, userId);
  const result = await agent.run(input, {
    sessionId,
    userId,
    env: c.env as any,
  });

  return c.json(result);
});

---

wrangler.toml — thêm R2

[[r2_buckets]]
binding = "R2"
bucket_name = "flare-agent-workspace"

---

Checklist

• Tạo R2 bucket: wrangler r2 bucket create flare-agent-workspace
• Tạo packages/workspace/
• Test upload + read file
• Test parse CSV và Markdown
• Test search sau khi index
• Test agent dùng workspace tools

---

Series hoàn tất — 11 bài, 11 packages

@flare-agent/types
@flare-agent/providers     — Groq, WorkersAI, Ollama
@flare-agent/memory        — KV, D1, Vectorize
@flare-agent/core          — Agent Loop, Tool Registry
@flare-agent/workflow      — Graph-based Workflow
@flare-agent/multi-agent   — Agent Network, Handoff
@flare-agent/observability — Tracing, Debugging
@flare-agent/channels      — Telegram, Web Chat
@flare-agent/workspace     — Filesystem, Search, Skills ← mới

Workspace cho phép agent không chỉ trả lời mà còn làm việc với files — đúng nghĩa một agent có context lâu dài.