The Market Data (Kline) Engine in MagicTradeBot is the core system responsible for fetching, caching, and processing price data (candles/klines) across all strategies, signal generators, analytics modules, and risk management systems.
It directly impacts:
- Signal accuracy
- Execution speed
- Strategy responsiveness
- API usage and rate-limit safety
In short, it is the data backbone of the entire trading system.
๐ What the Kline Engine Controls
The engine manages how market data flows through the bot using several configurable layers.
1๏ธโฃ Core Market Data Fetching
๐น refresh_interval
Controls how often the bot polls market data.
- Lower values โ Faster reaction time
- Higher values โ Lower API usage
Recommended:
- 2โ5 seconds โ Scalping / HFT
- 10โ30 seconds โ Swing / long-term
This directly affects how quickly the bot reacts to new price movements.
๐น max_kline_length
Defines how many historical candles are stored per symbol.
- Used by indicators and signal logic
- Too low โ Long-lookback strategies may break
- Too high โ Increased memory usage
Typical range: 50โ100 candles
๐น chunk_size
Number of symbols processed per update cycle.
- Higher โ Faster synchronization across many pairs
- Too high โ API bursts and potential rate limits
Balances speed and exchange safety.
๐น kline_refresh_rate
Controls full kline refresh timing.
- Balances real-time accuracy
- Helps prevent exchange rate-limit violations
Recommended: 5โ15 seconds for active trading
2๏ธโฃ Analytics & Signal Processing Layer
The engine separates signal creation from execution logic.
๐น summary_interval
Defines the base timeframe used for generating reports and summaries.
Example:
- "15m"
- "30m"
- "1h"
Used for analytics โ not direct trading execution.
๐น signal_refresh_interval
How often strategies generate signals.
- 20 seconds โ Fast scalping
- 1800 seconds โ Position trading
Controls signal frequency.
๐น decision_making_interval
How often the bot evaluates:
- Entries
- Exits
- Stop-loss
- Take-profit
This can be faster than signal generation, allowing rapid exit control even if signals update slower.
๐น screen_refresh_interval
Controls dashboard/UI refresh rate.
- Affects display only
- Does NOT impact trading logic
๐น analytic_watcher_interval
Defines how often periodic analytics summaries run.
- Recommended: 3600+ seconds (1 hour or more)
- Purely reporting-related
3๏ธโฃ Strategy Timeframe Mapping
The engine allows multi-timeframe support through:
strategy_intervals:
Each strategy type can use different candle intervals:
- hft โ 1m
- scalp โ Short-term intervals
- day โ Multi-timeframe confirmation
- swing โ Higher timeframes
- long โ Daily candles
- default โ Fallback configuration
This enables:
- Multi-timeframe confirmation
- Strategy specialization
- Exchange-specific interval formatting
4๏ธโฃ Rate Limit & API Safety Controls
To protect your exchange account:
๐น maxRequestsPerSecond
Maximum API calls allowed per second.
Must align with exchange limits.
๐น safeBuffer
Applies a safety margin.
Example:
- maxRequestsPerSecond: 10
- safeBuffer: 0.2
- Effective usage = 8 requests per second
Prevents burst-related bans and temporary lockouts.
โก Why the Market Data Engine Is Critical
Improper tuning can cause:
- Delayed signals
- Inaccurate indicators
- Missed trade opportunities
- Exchange rate limits
- Temporary API bans
Proper tuning improves:
- Execution precision
- Strategy stability
- Resource efficiency
- System scalability
๐ง Performance Principles
- Lower intervals โ Faster reaction, higher API load
- Higher intervals โ Less noise, lower resource usage
- Over-aggressive polling โ Risk of rate limits
- Under-aggressive polling โ Delayed trades
The ideal configuration depends on:
- Strategy type (HFT, scalp, swing, long-term)
- Number of symbols
- Exchange API limits
- Server performance
๐ Summary
The Market Data (Kline) Engine in MagicTradeBot:
- Controls how candles are fetched and stored
- Powers all strategies and risk systems
- Manages signal timing and execution cycles
- Protects against API rate-limit violations
- Balances speed, accuracy, and safety
It is the central nervous system of the trading infrastructure โ determining how quickly and accurately the bot sees and reacts to the market.