The Market Data (Kline) Engine in MagicTradeBot is the core system responsible for fetching, caching, and processing price data (candles/klines) across all strategies, signal generators, analytics modules, and risk management systems.
It directly impacts:
- Signal accuracy
- Execution speed
- Strategy responsiveness
- API usage and rate-limit safety
In short, it is the data backbone of the entire trading system.
🔎 What the Kline Engine Controls
The engine manages how market data flows through the bot using several configurable layers.
1️⃣ Core Market Data Fetching
🔹 refresh_interval
Controls how often the bot polls market data.
- Lower values → Faster reaction time
- Higher values → Lower API usage
Recommended:
- 2–5 seconds → Scalping / HFT
- 10–30 seconds → Swing / long-term
This directly affects how quickly the bot reacts to new price movements.
🔹 max_kline_length
Defines how many historical candles are stored per symbol.
- Used by indicators and signal logic
- Too low → Long-lookback strategies may break
- Too high → Increased memory usage
Typical range: 50–100 candles
🔹 chunk_size
Number of symbols processed per update cycle.
- Higher → Faster synchronization across many pairs
- Too high → API bursts and potential rate limits
Balances speed and exchange safety.
🔹 kline_refresh_rate
Controls full kline refresh timing.
- Balances real-time accuracy
- Helps prevent exchange rate-limit violations
Recommended: 5–15 seconds for active trading
2️⃣ Analytics & Signal Processing Layer
The engine separates signal creation from execution logic.
🔹 summary_interval
Defines the base timeframe used for generating reports and summaries.
Example:
- "15m"
- "30m"
- "1h"
Used for analytics — not direct trading execution.
🔹 signal_refresh_interval
How often strategies generate signals.
- 20 seconds → Fast scalping
- 1800 seconds → Position trading
Controls signal frequency.
🔹 decision_making_interval
How often the bot evaluates:
- Entries
- Exits
- Stop-loss
- Take-profit
This can be faster than signal generation, allowing rapid exit control even if signals update slower.
🔹 screen_refresh_interval
Controls dashboard/UI refresh rate.
- Affects display only
- Does NOT impact trading logic
🔹 analytic_watcher_interval
Defines how often periodic analytics summaries run.
- Recommended: 3600+ seconds (1 hour or more)
- Purely reporting-related
3️⃣ Strategy Timeframe Mapping
The engine allows multi-timeframe support through:
strategy_intervals:
Each strategy type can use different candle intervals:
- hft → 1m
- scalp → Short-term intervals
- day → Multi-timeframe confirmation
- swing → Higher timeframes
- long → Daily candles
- default → Fallback configuration
This enables:
- Multi-timeframe confirmation
- Strategy specialization
- Exchange-specific interval formatting
4️⃣ Rate Limit & API Safety Controls
To protect your exchange account:
🔹 maxRequestsPerSecond
Maximum API calls allowed per second.
Must align with exchange limits.
🔹 safeBuffer
Applies a safety margin.
Example:
- maxRequestsPerSecond: 10
- safeBuffer: 0.2
- Effective usage = 8 requests per second
Prevents burst-related bans and temporary lockouts.
⚡ Why the Market Data Engine Is Critical
Improper tuning can cause:
- Delayed signals
- Inaccurate indicators
- Missed trade opportunities
- Exchange rate limits
- Temporary API bans
Proper tuning improves:
- Execution precision
- Strategy stability
- Resource efficiency
- System scalability
🧠 Performance Principles
- Lower intervals → Faster reaction, higher API load
- Higher intervals → Less noise, lower resource usage
- Over-aggressive polling → Risk of rate limits
- Under-aggressive polling → Delayed trades
The ideal configuration depends on:
- Strategy type (HFT, scalp, swing, long-term)
- Number of symbols
- Exchange API limits
- Server performance
🚀 Summary
The Market Data (Kline) Engine in MagicTradeBot:
- Controls how candles are fetched and stored
- Powers all strategies and risk systems
- Manages signal timing and execution cycles
- Protects against API rate-limit violations
- Balances speed, accuracy, and safety
It is the central nervous system of the trading infrastructure — determining how quickly and accurately the bot sees and reacts to the market.