Riverd LWAS - Linux WEB Application Server

Riverd LWAS is a multi-language web application server with persistent and ephemeral workers, written in C. It allows efficient handling of HTTP requests through a worker system that can execute different programming languages.

Main Features

• Multi-language support: Can execute workers in any language that implements the defined communication protocol
• Persistent and ephemeral workers: The system can manage both types of workers for different usage scenarios
• Persistent workers: Workers remain in execution to process multiple requests, reducing process creation overhead
• Ephemeral workers: Workers that run for a single request and then close, useful for applications that don't need persistence
• Automatic worker management: The server monitors and recreates workers if they fail or close unexpectedly
• Support for HTTP/1.1 and HTTP/2: Can handle requests from different versions of the HTTP protocol
• Configurable logging system: Allows recording events, errors, and requests at different levels
• Flexible configuration: Can be configured through INI file or command-line parameters
• Resource limits: Control of simultaneous and per-IP connections, configurable timeouts

Architecture

The system is composed of:

• Master server (riverd.c): Coordinates worker creation, accepts client connections, and distributes requests among available workers
• Workers: Child processes that execute application logic and process HTTP requests
• Communication layer: Manages communication between master server and workers through socketpairs
• Protocol system: Handles HTTP request/response routing and adaptation
• Logging system: Records system events
• Resource control system: Manages connection limits and other resources

Installation and compilation

To compile Riverd LWAS:

make

Or using the build script:

./build.sh

Usage

Execution options

The server can be started with different configurations:

./riverd [-c config_file] [port] [worker_path] [num_workers]

Where:

• -c config_file: Path to configuration file (default: server_config.ini)
• port: Port to listen on (default: 8080)
• worker_path: Path to worker script (default: simple_php_worker.php)
• num_workers: Number of workers to use ('auto' for system cores, default: auto)

Usage examples

# Port 8080, default worker, auto workers
./riverd

# Port 8080, default worker, auto workers, with custom config
./riverd -c my_config.ini

# Port 9000, default worker, auto workers
./riverd 9000

Advanced Usage with Dummy Interpreter

Riverd LWAS includes a powerful system called the dummy_interpreter that allows running binary workers compiled from various programming languages. This system enables maximum flexibility and performance.

Dummy Interpreter System

The dummy_interpreter is located in the dummy_interpreter/ directory and acts as an abstraction layer between the Riverd server and binary workers. It allows the server to execute workers written in:

• C/C++ (compiled binaries)
• Pascal (Free Pascal compiled executables)
• Assembly (NASM compiled binaries)
• Any language that compiles to native binary

How it works:

1. The dummy interpreter receives the binary path and socket file descriptor as arguments
2. It executes the real binary worker using execv(), passing the socket file descriptor
3. The binary worker communicates directly with the client through the provided socket
4. This enables maximum performance as the worker runs as a native binary

Configuration example:

[workers]
binary_path = /home/bds/riverd-lwas/dummy_interpreter/dummy_interpreter
worker_path = /path/to/compiled_worker.bin

Pascal Workers (pascal-worker/ directory)

The pascal-worker/ directory contains examples and implementations of workers written in Pascal and compiled to native binaries using Free Pascal Compiler (FPC).

Features:

• Native Performance: Pascal workers compile to efficient native machine code
• Ephemeral Execution: Workers run once per request and terminate (perfect for stateless applications)
• Memory Efficiency: Direct memory management without interpretation overhead
• Full Integration: Works seamlessly with Riverd's request handling system

Included files:

• worker_pascal.pas: Main Pascal worker implementation
• pascal_worker_config.ini: Configuration file for Pascal workers
• README.md: Documentation about Pascal workers

C Workers (C/ directory)

The C/ directory contains C language workers that demonstrate efficient, low-level worker implementations.

Features:

• Maximum Performance: C workers offer the highest possible performance
• Low Resource Usage: Minimal overhead and memory usage
• Direct System Access: Full access to system calls and low-level operations
• Robust Implementation: Proven stability and efficiency

Included files:

• worker_en_c.c: C worker implementation
• Makefile: Build system for C workers
• worker-c.ini: Configuration file for C workers

Pascal Workers (pascal/ directory)

The pascal/ directory contains examples and implementations of workers written in Pascal and compiled to native binaries using Free Pascal Compiler (FPC).

Features:

• Native Performance: Pascal workers compile to efficient native machine code
• Ephemeral Execution: Workers run once per request and terminate (perfect for stateless applications)
• Memory Efficiency: Direct memory management without interpretation overhead
• Full Integration: Works seamlessly with Riverd's request handling system

Included files:

• worker_pascal.pas: Main Pascal worker implementation
• pascal_worker_config.ini: Configuration file for Pascal workers
• README.md: Documentation about Pascal workers

Benefits of Binary Workers

Using the dummy interpreter system with binary workers (C, Pascal) provides:

• Higher Performance: Up to 3x faster execution compared to interpreted languages
• Lower Memory Usage: No interpreter runtime overhead
• Better Security: Reduced attack surface compared to interpreted languages
• Flexibility: Support for any compiled language that follows the socket communication protocol
• Resource Efficiency: Direct memory and resource management
• Complete Control: Full access to system resources and operations

Port 9000, custom worker, auto workers

./riverd 9000 my_app_worker.php

Port 9000, custom worker, 8 workers

./riverd 9000 my_app_worker.php 8

Port 9000, custom worker, workers according to cores

./riverd 9000 my_app_worker.php auto

Port 9000, custom worker, 8 workers, with custom config

./riverd -c my_config.ini 9000 my_app_worker.php 8

## Configuration

The system can be configured through an INI file with the following sections:

### Section [server]
- `port`: Port the server will listen on

### Section [workers]
- `num_workers`: Number of workers to create (0 to use system cores)
- `binary_path`: Path to language binary (e.g., `/usr/bin/php`)
- `worker_path`: Path to worker file (e.g., `simple_php_worker.php`)
- `binary_flags`: Additional flags for language binary

### Section [logging]
- `enabled`: 1 to enable logging, 0 to disable
- `log_file`: Path to log file (default: `/tmp/phpwkd.log`)
- `level`: Logging level (debug, info, warn, error)

### Section [limits]
- `max_connections`: Maximum number of simultaneous connections (0 for unlimited)
- `client_timeout`: Timeout in seconds for client connections (default: 30)
- `connections_per_ip`: Maximum number of connections per IP (0 for unlimited)

### Section [http]
- `supported_versions`: Supported HTTP versions (e.g., "1.1,2.0")
- `keep_alive`: 1 to enable keep-alive, 0 to disable
- `max_requests_per_connection`: Maximum number of requests per keep-alive connection

## Communication protocol

The server communicates with workers using Unix socketpairs. Workers must implement the communication protocol to receive HTTP requests and return valid HTTP responses.

## Complementary Frameworks

RiverD LWAS can be used with specialized frameworks that facilitate the creation of workers:

### PyBlack Framework - Professional Framework for Python

PyBlack is a professional framework for creating ephemeral and persistent workers in the RiverD server, with full support for routing, middleware, and advanced abstractions.

#### PyBlack Features

- **Complete abstractions**: `Request` and `Response` classes with useful methods
- **Advanced routing system**: Support for parameterized routes and different HTTP methods
- **Middleware**: Middleware system for preprocessing requests
- **Route parameters**: Support for typed route parameters (e.g., `{user_id:int}`)
- **Content management**: Support for JSON, forms, and other content types
- **Robust error handling**: Built-in error handling system

#### PyBlack Components (`pyblack/`)

- `advanced_core.py` - Main WorkerFramework class with ephemeral/persistent mode support
- `request_response.py` - Request and Response classes with complete abstractions
- `router.py` - Advanced routing system with typed parameter support
- `middleware.py` - Middleware system for preprocessing requests
- `__init__.py` - Package initialization file
- `howto/` - Framework usage documentation

### BlackCoffee Framework - Professional Framework for PHP

BlackCoffee is a high-performance PHP framework specifically designed to work with Riverd LWAS. This framework enables the creation of high-performance web applications leveraging persistent PHP workers.

#### Framework Features

- **Flexible routing**: Supports parameterized routes and constraints
- **Middleware**: Middleware system for authentication, logging, etc.
- **Controllers**: Controller-based structure to organize business logic
- **Request/response management**: Complete API for manipulating HTTP requests and responses
- **High performance**: Leverages the server's persistent workers
- **Enhanced security**: Input validation and sanitization to prevent header injection, XSS, and other attacks
- **State cleanup**: Prevents information leaks between requests in persistent workers

#### Framework Structure

- `framework/Core/`: Contains the main framework classes
  - `Server.php`: Main server class
  - `Controller.php`: Base class for controllers
- `framework/Http/`: Classes for HTTP request and response handling
  - `HttpRequest.php`: Class for handling HTTP requests
  - `HttpResponse.php`: Class for handling HTTP responses
- `framework/Routing/`: Routing system
  - `Router.php`: Routing system with parameterized route support
- `framework/Middleware/`: Middleware components
  - `Middleware.php`: Base class for middlewares
  - `AuthMiddleware.php`: Example authentication middleware
- `framework/docs/`: Detailed framework documentation
  - `Routing.md`: Routing module documentation
  - `HttpRequest_HttpResponse.md`: Request and response documentation
- `framework/`: Main files
  - `PHPPWKF.php`: Main framework file (renamed as BlackCoffee)
  - `composer.json`: Dependency management configuration
  - `CHANGELOG.md`: Change log and framework evolution

## Worker Examples (`python/`)

The `python/` directory contains different worker examples demonstrating the framework's capabilities:

### Example Workers (`python/`)

#### 1. Advanced Ephemeral Worker
- **File**: `advanced_ephemeral_worker.py`
- **Description**: Processes a single request and terminates
- **Configuration**: `advanced_ephemeral_config.ini`

#### 2. Advanced Persistent Worker
- **File**: `advanced_persistent_worker.py`
- **Description**: Keeps the process active and processes multiple requests
- **Configuration**: `advanced_persistent_config.ini`

#### 3. REST API Example
- **File**: `rest_api_example.py`
- **Description**: Complete REST API example with user and post CRUD
- **Configuration**: `rest_api_config.ini`

#### 4. Persistent Services
- **File**: `persistent_services.py` (fixed version with separated responsibilities)
- **Description**: Complete API for products, users, and orders
- **Configuration**: `persistent_services_fixed_config.ini`

## Advanced Examples (`python/webExample/`)

The `python/webExample/` directory contains advanced examples demonstrating complex architectures and development best practices:

### Website in Ephemeral Mode with Separation of Responsibilities

- **Description**: Complete web application that consumes external services
- **Directory**: `python/webExample/`
- **Components**:
  - **Controllers**: Request/response coordination logic
  - **Services**: Business logic and data manipulation
  - **Utilities**: Reusable support functions
  - **Templates**: Presentation logic and views
  - **Main Worker**: Component binder

### Persistent Services with Separated Architecture

- **Description**: Complete REST API with clearly separated responsibilities
- **Location**: `python/webExample/persistent_service/`
- **Configuration file**: `python/webExample/persistent_services_config.ini`
- **Components**:
  - **Models**: Data structure and data access
  - **Controllers**: Coordination logic between HTTP layers and services
  - **Services**: Business logic and domain rules
  - **Utilities**: Reusable support components

### Ephemeral Web Site

- **Description**: Complete web application consuming external services
- **Location**: `python/webExample/`
- **Configuration file**: `python/webExample/web_example_config.ini`
- **Components**:
  - **Controllers**: Request/response coordination logic
  - **Services**: Business logic and data manipulation
  - **Utilities**: Reusable support functions
  - **Templates**: Presentation logic and views
  - **Main Worker**: Component binder

### Advanced Examples Features

- **Separation of responsibilities**: Well-structured code separated by responsibilities
- **Reusability**: Components reusable in different parts
- **Testing ease**: Each component can be tested individually
- **Scalability**: Easy to add new functionalities
- **Clarity**: More readable and structured code
- **Integration**: Efficient communication between components through the backend

## Communication Between Applications of Different Modes

One of the most important aspects of the framework is its ability to allow communication between applications operating in different modes. The `python/webExample/` directory demonstrates this key feature:

- **Ephemeral web site** (`python/webExample/`): Web application operating in ephemeral mode
- **Persistent services** (`python/webExample/persistent_service/`): REST API operating in persistent mode
- **Different ports**: The ephemeral web site runs on port 8084 while the persistent services run on port 8085
- **Internal communication**: The ephemeral web site consumes services from the persistent service via HTTP calls made from its backend
- **Microservices architecture**: Demonstrates how different components can communicate with each other while maintaining their optimal operation modes

This architecture allows:
- **Efficiency**: Ephemeral workers for frontend applications that don't need persistence
- **Performance**: Persistent workers for services that provide data or business logic
- **Scalability**: Each component can be scaled independently based on its needs
- **Flexibility**: Different components can be updated or modified without affecting others
- **Security**: Communication occurs in the backend, keeping internal service URLs hidden from the client

### Configuration Files

#### Specific Configurations
- `advanced_ephemeral_config.ini` - Configuration for ephemeral worker
- `advanced_persistent_config.ini` - Configuration for persistent worker
- `rest_api_config.ini` - Configuration for REST API example
- `persistent_services_config.ini` - Configuration for persistent services with separated responsibilities

#### Generic Configuration
- `pyblack_config.ini` - Generic example file that can be used as base

## Usage with RiverD

To use any of the workers with RiverD:

```bash
./riverd -c python/advanced_ephemeral_config.ini
# or
./riverd -c python/advanced_persistent_config.ini
# or
./riverd -c python/rest_api_config.ini
# or
./riverd -c python/webExample/persistent_services_config.ini
# or
./riverd -c python/webExample/web_example_config.ini

Examples of configuration and ephemeral workers

In the ephemeral/** directory you'll find examples of configuration and ephemeral workers in Python and PHP:

• ephemeral/: Contains configuration files for ephemeral workers
  • ephemeral_config.ini: Example configuration for an ephemeral worker in PHP
  • ephemeral_python_config.ini: Example configuration for an ephemeral worker in Python
• ephemeral/worker/: Contains the ephemeral worker files
  • ephemeral_worker.php: Example ephemeral worker in PHP
  • ephemeral_python_worker.py: Example ephemeral worker in Python

These examples show how to implement ephemeral workers that run once per request and then close, which is useful for applications that don't need persistence and require complete isolation between requests.

File Upload Controls

• Configurable Limits: The system allows configuring limits for file uploads from the INI file
• Configuration section: New [file_limits] section to control file uploads:
  • max_file_size: Maximum file size per individual file (configurable with MB/KB suffixes)
  • max_files_per_request: Maximum number of files per request
  • allowed_file_types: Allowed file types (comma-separated list)
  • validate_mime_types: Validate MIME types (1 to enable, 0 to disable)
• Default values: The system defines safe default values when no parameters are specified:
  • max_file_size: 10MB
  • max_files_per_request: 10 files
  • allowed_file_types: All types allowed
  • validate_mime_types: Disabled
• Appropriate response: If a request exceeds limits, the server responds immediately with HTTP 413 Payload Too Large or HTTP 400 Bad Request
• Route verification: The system verifies if the destination route exists before processing the request body

Security

• The system includes secure path validation to prevent directory traversal
• HTTP request validity is verified before processing
• Resource controls to prevent excessive consumption
• DoS Protection against file uploads: Configurable limits prevent denial-of-service attacks based on large file uploads
• Robust System: The server does not hang when receiving files that exceed configured limits

Author

Benjamin Sanchez Cardenas Email: bsanchezcardenas@gmail.com

License

Apache 2.0