Real-Time Signal Processing Engine for High-Speed Trading Automation

Overview

MagicTradeBot's real-time signal processing engine is designed to analyze thousands of trading symbols simultaneously with minimal latency. The system processes over 30+ different signal algorithms per symbol, combining historical kline data with live tick updates to detect profitable trading opportunities the moment they emerge.

Core Architecture

Multi-Symbol Processing Pipeline

The bot maintains a continuous processing pipeline that:

  • Loads and caches historical kline data (candlestick data) for each monitored symbol
  • Streams live tick data to update the most recent candle in real-time
  • Executes 30+ signal algorithms in parallel for each symbol
  • Filters signals based on enabled configurations
  • Triggers actions (order placement or broadcasting) when conditions match

Data Flow

Historical Kline Data → Cache Layer → Signal Processing Engine
         ↓                                      ↑
Live Tick Stream ──────────────────────────────┘
         ↓
Signal Algorithms (30+) → Enabled Filter → Action Router
         ↓                                      ↓
   Signal Output                    ┌───────────┴───────────┐
                                    ↓                       ↓
                            Order Placement          Broadcast
                            (if in volatility_       (if in volatility_
                             action list)            action_broadcast list)

Supported Signal Types (30+)

Trend & Momentum Signals

Basic Directional Signals

  • UP — Bullish movement detected
  • DOWN — Bearish movement detected

Volume & Price Action

  • PUMP — Sudden large upward volatility with volume surge
  • CRASH — Sudden large downward volatility with volume dump
  • SPIKE_PUMP — Extreme price spike upward (requires enable_spike_detection = true)
  • SPIKE_CRASH — Extreme price spike downward (requires enable_spike_detection = true)

Reversal & Recovery Signals

  • RECOVERY — Rapid rebound after a significant drop
  • REVERSAL — Confirmed trend direction reversal
  • STOP_HUNT_RECOVERY — Stop-loss hunt followed by recovery bounce
  • FISHER_STOP_HUNT_RECOVERY — Fisher Transform confirmed recovery
  • STOP_HUNT_REVERSAL — Stop-loss hunt followed by reversal
  • FISHER_STOP_HUNT_REVERSAL — Fisher Transform confirmed reversal

Accumulation & Distribution

  • ACCUMULATION — Sideways consolidation with stealth buying
  • FISHER_ACCUMULATION — Fisher Transform confirmed accumulation
  • DISTRIBUTION — Distribution phase indicating downtrend
  • FISHER_DISTRIBUTION — Fisher Transform confirmed distribution

Breakout Signals

  • BREAKOUT_UP / FISHER_BREAKOUT_UP
  • BREAKOUT_DOWN / FISHER_BREAKOUT_DOWN

Support & Resistance

  • SUPPORT_ABSORPTION — Price absorption at strong support
  • RESISTANCE_ABSORPTION — Price absorption near resistance

Technical Indicator Signals

RSI-Based

  • RSI_BUY — Oversold (requires enable_oversold_signal = true)
  • RSI_SELL — Overbought (requires enable_overbought_signal = true)

Volume Analysis

  • VOLUME_SPIKE_BUY
  • VOLUME_SPIKE_SELL

Momentum & Velocity

  • MOMENTUM_BUY, MOMENTUM_SELL
  • VELOCITY_BUY, VELOCITY_SELL

ATR, Bollinger, MACD, Divergence

  • ATR_BREAKOUT_BUY / ATR_BREAKOUT_SELL
  • BB_BUY / BB_SELL
  • MACD_BUY / MACD_SELL
  • DIVERGENCE_BUY / DIVERGENCE_SELL

Advanced Multi-Factor Signals

  • COMBINED_BUY / COMBINED_SELL
  • MTF_BUY / MTF_SELL
  • CVD_BUY / CVD_SELL
  • VWAP_BUY / VWAP_SELL
  • RS_BUY / RS_SELL
  • REGIME_BUY / REGIME_SELL
  • CHOP_BUY / CHOP_SELL
  • OFI_BUY / OFI_SELL
  • ICHIMOKU_BUY / ICHIMOKU_SELL

Long-Term Smart Money Signals

  • LONGTERM_SMART_LONG_SIGNAL
  • LONGTERM_SMART_SHORT_SIGNAL

Signal Processing Workflow

Step 1: Data Preparation

  1. Load historical kline data from cache or API
  2. Validate data completeness (gaps, missing candles)
  3. Merge latest tick data into active candle
  4. Calculate technical indicators (RSI, MACD, ATR, Bollinger, etc.)
  5. Compute derived metrics (volume delta, velocity, momentum)

Step 2: Signal Algorithm Execution

The engine runs all enabled algorithms concurrently. Each algorithm analyzes the prepared dataset and emits signals with: type, direction, strength, timestamp, and confidence score.

Step 3: Signal Filtering

Signals pass through multiple filters before action:

  • Enable/Disable Check — Signals must be enabled in config (e.g., SPIKE_CRASH needs enable_spike_detection).
  • Volatility Action Filter — Matches against supported_volatility_action for automated orders.
  • Broadcast Filter — Matches against supported_volatility_action_broadcast for notifications.

Step 4: Action Execution

Automated Order Placement

IF signal.type IN supported_volatility_action:
    → Place order based on signal direction
    → Apply risk management rules
    → Log order details

Signal Broadcasting

IF signal.type IN supported_volatility_action_broadcast:
    → Format signal data
    → Send to Discord / Telegram / WhatsApp / other channels

Configuration Example

# Order Placement Triggers
supported_volatility_action:
  - "BUY"
  - "SELL"
  - "PUMP"
  - "CRASH"

# Broadcast Triggers (wider set for notifications)
supported_volatility_action_broadcast:
  - "BUY"
  - "SELL"
  - "PUMP"
  - "CRASH"
  - "OFI_BUY"
  - "OFI_SELL"
  - "VOLUME_SPIKE_BUY"
  - "VOLUME_SPIKE_SELL"
  - "DIVERGENCE_BUY"
  - "DIVERGENCE_SELL"

Notes: After changing signal settings, re-sync symbols (remove from symbols.json and re-import). Use a broader broadcast list for monitoring without automated execution.

Performance Characteristics

Zero-Latency Design

  • In-memory caching to reduce API calls
  • Tick streaming updates only active candle
  • Parallel processing across symbols and algorithms
  • Optimized calculations that reuse intermediate values

Scalability

  • Handles thousands of symbols simultaneously
  • Processes 30+ algorithms per symbol in milliseconds
  • Supports multiple timeframes per symbol
  • Scales horizontally with extra processing nodes

Reliability

  • Fault tolerance — continues if individual signals fail
  • Data validation before processing
  • Graceful degradation — skips bad symbols
  • Error logging for debugging

Use Cases

  • High-Frequency Trading — Use SPIKE, PUMP, CRASH signals for rapid entries/exits
  • Smart Money Following — Track institutional patterns with LONGTERM_SMART signals
  • Multi-Strategy Portfolio — Run combinations across symbol groups
  • Risk Management — Use broadcast-only signals for manual review
  • Market Scanning — Monitor hundreds of symbols for rare opportunities

Best Practices

  1. Start Conservative — Enable fewer signals initially
  2. Separate Trading from Monitoring — Wider broadcast list than action list
  3. Backtest — Validate signal combos before live use
  4. Monitor Resources — Ensure sufficient compute for 30+ signals/symbol
  5. Review Logs — Track which signals perform best
  6. Adjust Thresholds — Tune sensitivity to market conditions

Technical Requirements

  • Stable WebSocket connection for tick streaming
  • Sufficient RAM for kline cache across symbols
  • Low-latency network to exchange APIs
  • Multi-core CPU for parallel processing
  • Persistent storage for signal history and analytics

📎 Related Topics