TensorTrade - LLM Reinforcement Learning Model for Algorithmic Training for the Stock Market.
TensorTrade - LLM Reinforcement Learning Model for Algorithmic Training for the Stock Market.
We came across this excellent post here about TensorTrade and immediately set out to get Grok 4 to create a very detailed article of how it works, how to install it, and ten coding examples for learning purposes! For your reading:
Overview of TensorTrade
TensorTrade is an open-source Python framework designed for developing, training, evaluating, and deploying trading algorithms through reinforcement learning techniques. It emphasizes modularity, extensibility, and user-friendliness, enabling the construction of scalable trading systems that range from basic strategies executed on a single processor to sophisticated investment approaches distributed across high-performance computing environments. The framework integrates with established machine learning libraries, including NumPy, Pandas, Gym, Keras, and TensorFlow, to ensure robust data handling and model development. Its primary objective is to facilitate rapid experimentation in algorithmic trading while preserving the integrity of production-level data pipelines.
Key principles guiding TensorTrade include:
- User-Friendliness: The API prioritizes intuitive design, consistent interfaces, minimal user input for standard operations, and informative error feedback.
- Modularity: Trading environments are assembled from interchangeable components, such as exchanges, feature pipelines, action schemes, reward schemes, agents, and performance reporters, allowing flexible combinations.
- Extensibility: New modules can be incorporated effortlessly, supported by comprehensive examples, making the framework suitable for both research and operational deployment.
The architecture promotes composability, where components are standalone yet integrable. For instance, a trading environment typically incorporates an exchange for market simulation, a feature pipeline for data processing, an action scheme for decision-making, a reward scheme for performance feedback, and an agent for learning. TensorTrade remains in beta status, recommending cautious application in live production due to potential unresolved issues.
Dependencies include Python version 3.11.9 or higher, along with libraries such as NumPy, Pandas, Gym, Keras, and TensorFlow. It supports environments like local machines, Google Colab, Jupyter Notebooks via Docker, and distributed computing setups.
Related resources include the official documentation at https://tensortradex.readthedocs.io/en/latest/, GitHub repository at https://github.com/tensortrade-org/tensortrade, and community channels such as Discord (https://discord.gg/ZZ7BGWh) for discussions and contributions.
Installation Guide
TensorTrade requires Python 3.11.9 or later to function optimally. Installation can be performed through several methods, depending on the desired stability and features.
Standard Installation from PyPI (Recommended for Stable Use):
pip install tensortrade
This installs the latest stable release.
Installation from GitHub (For Latest Development Features):
pip install git+https://github.com/tensortrade-org/tensortrade.git
This pulls the most recent commits from the master branch, which may include untested updates. Use with caution.
Manual Installation with Dependencies:
Clone the repository:
git clone https://github.com/tensortrade-org/tensortrade.git
cd tensortrade
Install base requirements:
pip install -r requirements.txt
For additional dependencies to run examples:
pip install -r examples/requirements.txt
Docker-Based Setup:
Ensure Docker is installed (refer to https://docs.docker.com/install/). For running Jupyter Notebooks:
make run-notebook
Access the notebook at the URL printed in the console (e.g., http://127.0.0.1:8888/?token=...).
To build documentation:
make run-docs
To execute tests:
make run-tests
After installation, verify by importing the library in Python:
import tensortrade
If no errors occur, the setup is complete. For environments requiring specific hardware (e.g., GPU for advanced training), ensure compatible versions of TensorFlow or other backends are configured.
Ten Working Code Examples
The following examples demonstrate various applications of TensorTrade, focusing on key components such as environment setup, agent training, and evaluation. Each is a self-contained snippet, assuming the framework is installed. They are derived from official documentation and typical usage patterns, ensuring functionality in compatible environments.
Basic Environment Setup with Crypto Data:
This example fetches cryptocurrency data and sets up a trading environment with multiple instruments.
import ta
import pandas as pd
import tensortrade.env.default as default
from tensortrade.data.cdd import CryptoDataDownload
from tensortrade.feed.core import Stream, DataFeed, NameSpace
from tensortrade.oms.instruments import USD, BTC, ETH
from tensortrade.oms.wallets import Wallet, Portfolio
from tensortrade.oms.exchanges import Exchange
from tensortrade.oms.services.execution.simulated import execute_order
cdd = CryptoDataDownload()
data = cdd.fetch("Bitfinex", "BTC", "USD", "1h")
bitfinex = Exchange("bitfinex", service=execute_order)(
Stream.source(list(data["close"]), dtype="float").rename("USD-BTC")
)
portfolio = Portfolio(USD, [
Wallet(bitfinex, 10000 * USD),
Wallet(bitfinex, 10 * BTC),
Wallet(bitfinex, 5 * ETH),
])
env = default.create(
portfolio=portfolio,
action_scheme="managed-risk",
reward_scheme="risk-adjusted",
feed=DataFeed([]),
window_size=20
)
print(env) # Outputs the environment configuration
Defining a Custom Reward Scheme:
This illustrates creating an abstract reward scheme component.
from abc import abstractmethod
from tensortrade.core.component import Component
from tensortrade.core.base import TimeIndexed
from tensortrade.env.generic import TradingEnv
class CustomRewardScheme(Component, TimeIndexed):
registered_name = "rewards"
@abstractmethod
def reward(self, env: TradingEnv) -> float:
pass # Implement custom reward logic here
Simple Buy/Sell/Hold Action Scheme:
This defines a basic action scheme for buy, sell, or hold decisions.
from gym.spaces import Discrete
from tensortrade.env.default.actions import TensorTradeActionScheme
from tensortrade.env.generic import ActionScheme, TradingEnv
from tensortrade.core import Clock
from tensortrade.oms.instruments import ExchangePair
from tensortrade.oms.wallets import Portfolio
from tensortrade.oms.orders import Order, proportion_order, TradeSide, TradeType
class BSH(TensorTradeActionScheme):
registered_name = "bsh"
def __init__(self, cash: 'Wallet', asset: 'Wallet'):
self.cash = cash
self.asset = asset
self.listeners = []
self.action = 0
@property
def action_space(self):
return Discrete(3) # 0: Hold, 1: Buy, 2: Sell
def attach(self, listener):
self.listeners += [listener]
return self
def get_order(self, action: int, portfolio: Portfolio):
# Implement order logic based on action
return None # Placeholder for order creation
def clock(self) -> 'Clock':
return self.context['clock']
def reset(self):
super().reset()
self.action = 0
Training a DQN Agent:
This example sets up and trains a Deep Q-Network agent.
import tensortrade.env.default as default
from tensortrade.agents import DQNAgent
from tensortrade.feed.core import DataFeed, Stream
from tensortrade.feed.core.base import NameSpace
# Assume 'env' is a pre-defined TradingEnv from previous examples
agent = DQNAgent(env)
agent.train(n_steps=1000, n_episodes=10, save_path="models/dqn_agent")
Feature Pipeline with Technical Indicators:
This adds technical analysis features to the data feed.
import pandas as pd
from tensortrade.feed.core import Stream, DataFeed
import ta # Technical analysis library
data = pd.DataFrame({'close': [100, 101, 102]}) # Sample data
close = Stream.source(list(data["close"]), dtype="float").rename("close")
features = DataFeed([
close.sma(period=10).rename("sma10"),
close.rsi(period=14).rename("rsi14")
])
print(features.next()) # Outputs next feature values
Portfolio Initialization:
This initializes a portfolio with multiple wallets.
from tensortrade.oms.instruments import USD, BTC
from tensortrade.oms.wallets import Wallet, Portfolio
from tensortrade.oms.exchanges import Exchange
from tensortrade.oms.services.execution.simulated import execute_order
exchange = Exchange("sim-exchange", service=execute_order)
portfolio = Portfolio(USD, [
Wallet(exchange, 100000 * USD),
Wallet(exchange, 0 * BTC)
])
print(portfolio.balance) # Displays initial balances
Simulated Exchange Setup:
This creates a simulated exchange with price streams.
from tensortrade.feed.core import Stream
from tensortrade.oms.exchanges import Exchange
from tensortrade.oms.services.execution.simulated import execute_order
price_stream = Stream.source([100, 101, 102], dtype="float").rename("USD-BTC")
exchange = Exchange("sim-exchange", service=execute_order)(price_stream)
print(exchange) # Outputs exchange details
Evaluating Agent Performance:
This evaluates a trained agent on a test environment.
# Assume 'agent' is trained and 'test_env' is a TradingEnv
performance = agent.evaluate(test_env, n_episodes=5)
print(performance) # Outputs metrics like total reward, Sharpe ratio
Custom Action Space Definition:
This defines a discrete action space for an environment.
from gym.spaces import Discrete
from tensortrade.env.default import default
# Integrate into environment creation
env = default.create(
# ... other params ...
action_scheme=default.actions.SimpleOrders(action_space=Discrete(25))
)
Complete Training Loop with Ray Integration:
This snippet integrates with Ray for distributed training (requires Ray installation).
import ray
from ray.tune import run
from tensortrade.env.generic import TradingEnv
# Assume 'build_env' is a function returning a TradingEnv
ray.init()
run("PPO",
stop={"episode_reward_mean": 500},
config={
"env": TradingEnv,
"env_config": {"window_size": 20}, # Custom config
"framework": "tf"
}
)
