ObjectFS Architecture Evolution: FUSE Client → Multi-Protocol Server¶

This is a design sketch, not a description of the code. Nothing on the multi-protocol path is implemented: there is no SMB server, no NFS server, and no internal/protocols directory. Read it as one proposal for how a second protocol could be added.

Two things in it are also out of date, and are left in place rather than quietly rewritten because the divergence is the useful part:

The Phase 1 file layout proposes internal/filesystem + internal/protocols/fuse. The refactor that actually happened produced internal/vfs (POSIX semantics, no FUSE dependency) with internal/fuse as a thin go-fuse shim over it. That is the same idea arriving under a different name, and it is the layer a second protocol would bind to.

internal/filesystem/interface.go exists and declares a fuller surface than internal/vfs implements, but its only implementation is a mock. It is a contract, not a component.

The HTTP-handler column of the Phase 2 diagram shows "REST API". There was a pkg/api declaring 12 routes; it was deleted in #367 for having no importer, and a running mount serves only /metrics and /health. The diagram is not redrawn, because a proposal for a WebDAV/REST surface is a reasonable thing to propose — but nothing in the tree corresponds to that box now, which is the difference between this file and a description.

Multi-protocol work is tracked in #181 (SMB, then NFS) and #188 (NFS loopback on macOS). The phase-to-version mapping below (v0.3.0, v0.4.0, v0.5.0) is from 2025 and was not followed.

Current Architecture¶

┌─────────────────────────────────────────────────────────────────┐
│                    User Applications                            │
│  (cp, ls, grep, analysis tools, IDEs, etc.)                   │
└─────────────────────┬───────────────────────────────────────────┘
                      │ Standard POSIX calls
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                  Operating System                              │
│              (Linux/macOS/Windows)                             │
└─────────────────────┬───────────────────────────────────────────┘
                      │ FUSE protocol
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                  ObjectFS FUSE                                 │
│                    (Client)                                    │
└─────────────────────┬───────────────────────────────────────────┘
                      │ Direct function calls
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                ObjectFS Core Engine                            │
├─────────────────────┬───────────────────────────────────────────┤
│   S3 Backend        │  Cost Optimizer  │  Pricing Manager      │
│   ├─ CargoShip xport│  ├─ Tier Analysis │  ├─ Enterprise       │
│   ├─ Tier Management│  ├─ Access Patterns│  │   Discounts       │
│   └─ Multi-region   │  └─ Optimization  │  └─ Volume Pricing   │
├─────────────────────┼───────────────────┼───────────────────────┤
│             Cache System                │      Metrics          │
│             ├─ Multi-level LRU         │      ├─ Performance    │
│             ├─ Write buffering         │      ├─ Cost tracking  │
│             └─ Intelligent prefetch    │      └─ Usage analytics│
└─────────────────────┬───────────────────────────────────────────┘
                      │ AWS SDK calls
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                     AWS S3                                     │
│  (Multiple tiers: Standard, IA, Glacier, etc.)               │
└─────────────────────────────────────────────────────────────────┘

Target Architecture (v1.0+): Multi-Protocol Server¶

┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│  FUSE Clients   │ │  Windows SMB    │ │   macOS SMB     │ │   NFS Clients   │
│                 │ │    Clients      │ │    Clients      │ │    (future)     │
│ Linux/macOS/Win │ │  (Explorer,     │ │ (Finder, etc.)  │ │                 │
│                 │ │   Office, etc.) │ │                 │ │                 │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘
         │ FUSE              │ SMB 2.1/3.x        │ SMB 2.1/3.x        │ NFS v4
         │ protocol          │ protocol            │ protocol            │ protocol
         ▼                   ▼                     ▼                     ▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│                        ObjectFS Multi-Protocol Server                          │
├─────────────────────────────────────────────────────────────────────────────────┤
│                          Protocol Handler Layer                                │
├─────────────┬─────────────┬─────────────┬─────────────┬─────────────────────────┤
│FUSE Handler │ SMB Handler │NFS Handler  │HTTP Handler │   Future Protocols      │
│             │             │  (future)   │  (WebDAV)   │   (S3 API, etc.)       │
│├─Mount Mgmt │├─Auth & ACL │├─Export Mgmt│├─REST API   │                         │
│├─POSIX Ops  │├─Share Mgmt │├─Lock Mgmt  │├─Browser UI │                         │
│└─Permissions│└─Win Compat │└─Permissions│└─Mobile App │                         │
└─────────────┴─────────────┴─────────────┴─────────────┴─────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│                      Common Filesystem Interface                               │
│                                                                                 │
│  interface FilesystemInterface {                                               │
│    Open(path string, flags int) (FileHandle, error)                           │
│    Read(fh FileHandle, offset int64, size int) ([]byte, error)                │
│    Write(fh FileHandle, offset int64, data []byte) error                      │
│    ReadDir(path string) ([]DirEntry, error)                                   │
│    Stat(path string) (FileInfo, error)                                        │
│    // ... all filesystem operations                                           │
│  }                                                                             │
└─────────────────────┬───────────────────────────────────────────────────────────┘
                      │ UNCHANGED - Complete Backend Compatibility
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│                     ObjectFS Core Engine (UNCHANGED)                           │
├─────────────────────┬───────────────────┬───────────────────────────────────────┤
│   S3 Backend        │  Cost Optimizer   │  Pricing Manager                      │
│   ├─ CargoShip xport│  ├─ Tier Analysis │  ├─ Enterprise Discounts             │
│   ├─ Tier Management│  ├─ Access Pattern │  ├─ Institutional Config             │
│   ├─ Multi-region   │  │   Monitoring    │  ├─ Volume Pricing Tiers            │
│   └─ All S3 tiers   │  └─ Optimization   │  └─ Cost Calculations               │
├─────────────────────┼───────────────────┼───────────────────────────────────────┤
│             Cache System (ENHANCED)     │           Metrics (ENHANCED)          │
│             ├─ Multi-level LRU          │           ├─ Per-protocol stats       │
│             ├─ Protocol-aware caching   │           ├─ Multi-client monitoring   │
│             ├─ Write buffering          │           ├─ Cost tracking             │
│             └─ Smart prefetching        │           └─ Usage analytics           │
└─────────────────────┬───────────────────────────────────────────────────────────┘
                      │ UNCHANGED - AWS SDK calls
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│                              AWS S3                                            │
│     (All storage tiers with enterprise pricing and cost optimization)         │
└─────────────────────────────────────────────────────────────────────────────────┘

Key Architectural Principles¶

1. Zero Backend Disruption¶

// Current ObjectFS core remains 100% unchanged
type Backend struct {
    s3Client          *s3.Client
    costOptimizer     *CostOptimizer
    pricingManager    *PricingManager  
    cache            *cache.MultiLevel
    // ... all existing fields unchanged
}

// All existing methods work identically
func (b *Backend) PutObject(ctx context.Context, key string, data []byte) error
func (b *Backend) GetObject(ctx context.Context, key string) ([]byte, error)
func (b *Backend) ListObjects(ctx context.Context, prefix string) ([]ObjectInfo, error)
// ... all existing functionality preserved

2. Protocol Abstraction Interface¶

// New abstraction layer - protocols implement this
type FilesystemInterface interface {
    // File operations
    Open(ctx context.Context, path string, flags int) (FileHandle, error)
    Read(ctx context.Context, fh FileHandle, offset int64, size int) ([]byte, error)
    Write(ctx context.Context, fh FileHandle, offset int64, data []byte) error
    Close(ctx context.Context, fh FileHandle) error
    Flush(ctx context.Context, fh FileHandle) error

    // Directory operations
    ReadDir(ctx context.Context, path string) ([]DirEntry, error)
    Mkdir(ctx context.Context, path string, mode os.FileMode) error
    Rmdir(ctx context.Context, path string) error
    Remove(ctx context.Context, path string) error
    Rename(ctx context.Context, oldPath, newPath string) error

    // Metadata operations
    Stat(ctx context.Context, path string) (FileInfo, error)
    Chmod(ctx context.Context, path string, mode os.FileMode) error
    Chown(ctx context.Context, path string, uid, gid int) error
    Utimes(ctx context.Context, path string, atime, mtime time.Time) error

    // Extended operations
    Truncate(ctx context.Context, path string, size int64) error
    Link(ctx context.Context, oldPath, newPath string) error
    Symlink(ctx context.Context, target, linkPath string) error
    Readlink(ctx context.Context, path string) (string, error)
}

// Backend adapter implements the interface
type FilesystemBackend struct {
    backend *s3.Backend  // Existing backend unchanged
}

func (fb *FilesystemBackend) Open(ctx context.Context, path string, flags int) (FileHandle, error) {
    // Translate to existing backend.GetObject() calls
    // All cost optimization, pricing, caching work unchanged
}

3. Protocol Handler Pattern¶

type ProtocolHandler interface {
    Name() string
    Start(ctx context.Context) error
    Stop(ctx context.Context) error
    Metrics() ProtocolMetrics
}

// FUSE implementation (refactored from current code)
type FUSEHandler struct {
    fs          FilesystemInterface
    mountPoint  string
    fuseServer  *fuse.Server
    // ... FUSE-specific fields
}

// SMB implementation (new)
type SMBHandler struct {
    fs         FilesystemInterface
    config     SMBConfig
    server     *smb.Server
    auth       AuthProvider
    shares     []SMBShare
}

// Future NFS implementation  
type NFSHandler struct {
    fs         FilesystemInterface
    config     NFSConfig
    server     *nfs.Server
    exports    []NFSExport
}

4. Configuration Evolution¶

# Current configuration - remains unchanged
backends:
  s3:
    bucket: "research-data"
    region: "us-west-2"
    storage_tier: "STANDARD_IA"
    pricing_config:
      discount_config_file: "institutional-discounts.yaml"
    cost_optimization:
      enabled: true
    # ... all existing config unchanged

# NEW: Multi-protocol configuration (v0.3.0+)
protocols:
  fuse:
    enabled: true
    mount_point: "/mnt/s3-data"
    # All current FUSE config moves here

  smb:
    enabled: true
    listen_addr: ":445"
    server_name: "RESEARCH-DATA"
    workgroup: "UNIVERSITY"

    # Authentication
    auth_mode: "ldap"
    ldap_config:
      server: "ldap://directory.university.edu:389"
      bind_dn: "cn=objectfs,ou=services,dc=university,dc=edu"
      user_base: "ou=users,dc=university,dc=edu"

    # Shares mapping to S3 prefixes
    shares:
      - name: "genomics"
        path: "/genomics/"
        description: "Genomics Research Data"
        allowed_groups: ["bio-researchers", "lab-admins"]
        read_only: false

      - name: "reference"
        path: "/reference/"
        description: "Reference Genomes (Read-Only)"
        guest_access: true
        read_only: true

    # SMB protocol settings
    smb_versions: ["2.1", "3.0", "3.1.1"]
    encryption: true
    signing: true

  nfs:
    enabled: false  # Future implementation
    # NFS config here

Implementation Strategy¶

Phase 1: Refactoring (v0.3.0)¶

Goal: Prepare architecture without breaking existing functionality

Changes Required¶

Extract Common Interface: Create FilesystemInterface
Wrap Existing Backend: Create FilesystemBackend adapter
Refactor FUSE Code: Make it use the common interface
Update Configuration: Support protocol-specific config sections
Comprehensive Testing: Ensure zero regression

File Structure Changes¶

internal/
├── filesystem/           # New: Common interface
│   ├── interface.go     # FilesystemInterface definition
│   └── backend.go       # FilesystemBackend adapter
├── protocols/           # New: Protocol handlers
│   ├── fuse/           # Refactored FUSE code
│   │   ├── handler.go  
│   │   └── mount.go
│   ├── smb/            # Future SMB implementation
│   └── nfs/            # Future NFS implementation
└── storage/s3/         # UNCHANGED
    ├── backend.go      # All existing code unchanged
    ├── pricing_manager.go
    ├── cost_optimizer.go
    └── ...

Phase 2: SMB Implementation (v0.4.0)¶

Goal: Add basic SMB support with local authentication

Key Components¶

SMB Protocol Library Integration
Basic Authentication (local users)
Share Management
Windows Client Testing

Phase 3: Enterprise SMB (v0.5.0)¶

Goal: Production-ready with enterprise authentication

Key Features¶

LDAP/Active Directory Integration
Advanced ACLs and Permissions
SMB 3.x Encryption
Multiple Share Support

Benefits Analysis¶

Technical Benefits¶

Backend Compatibility: Zero disruption to existing S3/cost optimization
Code Reuse: All caching, pricing, optimization shared across protocols
Unified Configuration: Single config file manages all protocols
Consistent Behavior: Same S3 tier logic applies to all protocols

Market Benefits¶

Windows Market: Access to Windows-heavy enterprises
NAS Replacement: Position as S3-backed enterprise file server
Protocol Flexibility: Customers choose best protocol for their needs
Competitive Moat: Only multi-protocol S3 filesystem with cost intelligence

Operational Benefits¶

Single Deployment: One ObjectFS instance serves multiple protocols
Unified Monitoring: Single metrics/logging system for all protocols
Simplified Management: IT teams manage one system, not multiple tools
Cost Optimization: Enterprise pricing intelligence across all access methods

Risk Mitigation¶

Technical Risks¶

Complexity: Mitigate with careful phased approach and extensive testing
Performance: Ensure common interface doesn't introduce overhead
Protocol Compatibility: Invest in comprehensive client testing
Security: Implement enterprise-grade authentication and encryption

Business Risks¶

Market Timing: Validate enterprise SMB demand before heavy investment
Competition: Monitor for competitive responses and differentiate aggressively
Resource Requirements: Significant engineering investment required
Support Complexity: Multi-protocol support increases support burden

Success Criteria¶

Phase 1 (v0.3.0)¶

Zero performance regression for existing FUSE functionality
Clean architectural separation between protocols and backend
All existing tests pass unchanged
Configuration migration path documented

Phase 2 (v0.4.0)¶

Windows clients can connect via SMB
Basic file operations work (read, write, directory listing)
Performance within 80% of direct S3 access
Local authentication system functional

Phase 3 (v0.5.0)¶

LDAP/AD authentication working
Multiple shares with different permissions
SMB 3.x encryption and security features
Production deployment at 5+ enterprise customers

Conclusion: This multi-protocol architecture maintains complete backend compatibility while positioning ObjectFS as a unique, enterprise-grade, S3-backed file server. The phased approach minimizes risk while maximizing market impact.