Investor – Automatic Trading with E-Trade

Hurray – Today I finally managed to finish the implementation of the integration into E-Trade. Unfortunately it turned out to be a little bit more complicated then initially thought and there were quite a few stumbling blocks around the topic of “authentication”.

I have aleady demonstrated how to do trading with actual data.
In this document I want to show how to do automatic trading with E-Trade. Information on how to request to access E-Trade can be found in the official documentation

Setup

First – as always – we add the necessary jars and import all relevant packages:

%classpath config resolver maven http://192.168.1.10:8081/repository/maven-public/
%classpath add mvn ch.pschatzmann investor 0.9-SNAPSHOT
%classpath add mvn ch.pschatzmann investor-etrade 0.0.1-SNAPSHOT
%classpath add mvn ch.pschatzmann jupyter-jdk-extensions 0.0.1-SNAPSHOT

// our investor framwork
import ch.pschatzmann.dates._
import ch.pschatzmann.stocks._
import ch.pschatzmann.stocks.data.universe._
import ch.pschatzmann.stocks.input._
import ch.pschatzmann.stocks.accounting._
import ch.pschatzmann.stocks.accounting.kpi._
import ch.pschatzmann.stocks.execution._
import ch.pschatzmann.stocks.execution.fees._
import ch.pschatzmann.stocks.execution.price._
import ch.pschatzmann.stocks.parameters._
import ch.pschatzmann.stocks.strategy._
import ch.pschatzmann.stocks.strategy.optimization._
import ch.pschatzmann.stocks.strategy.allocation._
import ch.pschatzmann.stocks.strategy.selection._
import ch.pschatzmann.stocks.integration._
import ch.pschatzmann.stocks.integration.ChartData.FieldName._
import ch.pschatzmann.stocks.strategy.OptimizedStrategy.Schedule._

// e-trade
import ch.pschatzmann.investor.etrade._

// java
import java.util.stream.Collectors
import java.util._
import java.lang._
import java.util.function.Consumer
import scala.collection.JavaConverters

/// jupyter custom displayer
import ch.pschatzmann.display._

Added new repo: maven


import ch.pschatzmann.dates._
import ch.pschatzmann.stocks._
import ch.pschatzmann.stocks.data.universe._
import ch.pschatzmann.stocks.input._
import ch.pschatzmann.stocks.accounting._
import ch.pschatzmann.stocks.accounting.kpi._
import ch.pschatzmann.stocks.execution._
import ch.pschatzmann.stocks.execution.fees._
import ch.pschatzmann.stocks.execution.price._
import ch.pschatzmann.stocks.parameters._
import ch.pschatzmann.stocks.strategy._
import ch.pschatzmann.stocks.strategy.optimization._
import ch.pschatzmann.stocks.strategy.allocation._
import ch.pschatzmann.stocks.strategy.selection._
import ch.pschatzmann.stocks.integration._
import ch.pschatzmann.stocks.integration.ChartData.FieldName._
import ch.pschatzmann.stocks.strategy.OptimizedStrategy.Schedule._
import ch.pschatzmann.i...

Output

Unfortunatly the functionality provided by e-trade is printing information to System.out and therfore this will appear in our Notebook.

We switch this off by setting the standard output to the NullPrintScream:

System.setOut(new NullPrintStream())

null

Authentication / Login

In order to be able to login we need to have the following input defined in the ‘investor.properties’ file:
– EtradeUser
– EtradePassword
– EtradeConsumerKey
– EtradeConsumerSecret
– EtradeConsumerKeySandbox
– EtradeConsumerSecretSandbox

The user and password is the same information which is used to login to e-trade via the web-gui. The consumer_key and consumer_secret need to be requested from e-trade separatly.

The ETradeLogin class provides an authenicated clientRequest which has a valid access token associated with it. In the constructor we can indicated if we want to access the Production environemnt.
ETradeLogin is needed as input which goes into the ETradeTrader class.

After we are done we can revoke the access again.

// login to the Sandobx
var login = new ETradeLogin(false)
// list all available sandbox accounts
var accounts = login.getAccounts()
login.logoff()

accounts

[30049872, 83405188, 83405553, 83405557, 83406373, 83412346, 83412345, 83491757]

Trading with E-Trade

Instead of the PaperTrader we just use the ETradeTrader which makes sure that transactions are sent to e-trade and we monitor the the execution by e-trade so that we can update the status of our orders.

Here is a complete example:

 class MyScheduledExecutor() extends ScheduledExecutor() {
    var currentYear:Integer = null;

    override def getExecutor: StrategyExecutor = {
        var year = Context.getYear(new Date())
        var executor = super.getExecutor()
        if (year!=currentYear) {
            System.out.println("Defining new executor");
            var periods = Context.getDateRanges((year-2)+"-01-01",(year-1)+"-01-01");
            var account = new ManagedAccount("30049872", "USD", 0.00, Context.date("2010-01-01"), new ETradeFees())
            var login = new ETradeLogin(false)
            var etradeTrader = new ETradeTrader(login, account);            
            var allocationStrategy = new DistributedAllocationStrategy(etradeTrader);
            executor = new StrategyExecutor(etradeTrader, allocationStrategy);
            var portfolioUniverse =  new EdgarUniverse(year-2, Arrays.asList(0.2, 0.5, 0.8, 1.0), 10, Arrays.asList("NetIncomeLoss"), true)                                
            var strategies = TradingStrategyFactory.list()
            var strategySelector = new StrategySelector(account, strategies, periods.get(0), KPI.AbsoluteReturn)
            var stockSelector = new StockSelector(strategySelector)
            var reader = new YahooReader() 
            var result = stockSelector.getSelection(10, portfolioUniverse, reader)
            executor.setStrategies(result.getStrategies(reader));
            setExecutor(executor)
            currentYear = year
        }
        return executor
    }
 }

// schedule the execution every day at 9am
var scheduledExecutor = new MyScheduledExecutor()
scheduledExecutor.schedule("0 0 9 * * ?"); 

null

Synronization of E-Trade Information

var account = new ManagedAccount("83412345", "USD", 0.0, Context.date("2010-01-01"), new ETradeFees())
var login = new ETradeLogin(false)
var etradeTrader = new ETradeTrader(login, account);            

Displayers.display(account.getTransactions())

Synronization of Account

If we indicate no starting amount in the account, the system automatically synchronizes the account information with
e-trade in order to set up the actual as is information.

We can also explicitly request this by calling trader.synchronizeAccount() or schedule a automatic synchronization
by calling trader.scheduleSynchronizeAccount(cronExpression).

Any long stock which is available in the e-trade account will be replicated into our local account but it will not be considered by the trader.

We can easly achieve this by adding the missing stocks into the portfilioUniverse:

// synronize account to get the stocks from e-trade
etradeTrader.synchronizeAccount()
// build combined universe
var universe =  new ListUniverse()
universe.add(new AccountUniverse(account))
universe.add(new EdgarUniverse(year-2, Arrays.asList(0.2, 0.5, 0.8, 1.0), 10, Arrays.asList("NetIncomeLoss"), true))

Or we could just add it with a predefined fixed strategy

// synronize account to get the stocks from e-trade
etradeTrader.synchronizeAccount()
var fixedStrategies = Context.getStrategies(new AccountUniverse(account).getList(), reader, "BuyAndHoldStrategy")
executor.setStrategies(result.getStrategies(reader), fixedStrategies);

Synronization of Transactions

The default setup is automatically synchronizing the orders with the transactions every full hour.
The synchonization is implementing the following logic:
– If we find an executed order but there is no corresponsing transaction -> we create the missing transaction
– If for a transaction the order has been completed -> we complete the transaction
– If for a transaction the order has been rejected, cancelled or expired -> we cancel the transaction

We can explicity define a schedule in the constructor

var etradeTrader = new new ETradeTrader(account, Environment.SANDBOX, "key", "secret", cronSchedule);            

or we can just reschedule by calling

etradeTrader.scheduleSynchronizeTransactions(cronSchedule)

The scheduling can be canelled by passing an empty schedule.

Cron Syntax

The detailed description of the quartz cron syntax can be found at http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/crontrigger.html. E.g. ff we an it to run every 10 minutes we would specify “0 0/10 * * * ?”

Result

The latest updated account information can be found in the ‘accounts’ directory.
We can also determine the trasactions from the account which is available in the executor:

var account = new ManagedAccount("30049872", "USD", 0.00, Context.date("2010-01-01"), new ETradeFees())
Displayers.display(account.getTransactions())