Investor – Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator)

A Long short-term memory (LSTM) network is a special type of a recurrent neural network (RNN). It remembers values over arbitrary time intervals which makes it suited to be used for forecasting of time series. I was struggling with the DL4J implementation of the LSTM to be used to forecast stock data and I was not sure where the issue was: was the data not suited for forecasting ?. did I have a bug in my Iterator which was Read more…

By pschatzmann, ago

Investor – HarmonicStockOscillator¶

I thought it might be cool to have a stock price generator which is generating a steady sinus output, so that it can be used for the testing of trading strategies. A sinus generator function generally needs the following input amplitude omega phase This is oscillating between -amplitude and +amplitude so we also need to add the following additional parameter to raise the center on the y axis offset These parameter can also be determined Read more…

By pschatzmann, ago

Investor and Machine Learning (MLlib and DL4J)

In this Blog we demonstrate how Investor can be used together with DL4j and Spark’s MLlib. In general the idea is to use the stock history data in order to predict an increase or decrease of the stock value with the help of Machine Learning. We will demonstrate how we can use Indicators to define the machine learning model and how to evaluate the models and finally to check the result using the Investor framework. Here Read more…

By pschatzmann, ago

Investor – Portfolio Optimization

Portfolio optimization is the process of selecting the best portfolio (asset distribution), out of the set of all portfolios being considered, according to some objective. The objective typically maximizes factors such as expected return, and minimizes costs like financial risk. In this Blog we demonstrate how the Invstor API can be used to determine an optimized ‘Strategy Portfolios’. We use the RandomDistributor to generate randomly distributed porfolios and the KPIValues in order to determine the best combination. Read more…

By pschatzmann, ago

H2O Sparkling Water – Distributed Random Forrest

H2O is a machine learning framework which has been implemented in Java and provides an API for Scala, Python and R. This framework has the following unique features: – It has a interactive web GUI (Flow) so that you can work w/o any programming – The generated trained models can be deployed easily in a production JVM environment with minimal dependencies. In this document we demonstrate how H2O can be used with Scala & Spark Read more…

By pschatzmann, ago

Naive Bayes with Weka

If you are interested in Machine Learning in the JVM you should not forget about the good old Weka. It has basically been desinged to be used by a Swing GUI but it can also be used as an API. In terms of documentation I can recommend this manual and the javadoc. In my Demo I use the NaiveBayesMultinomial classifier with the iris dataset that is directly loaded from the Internet. The Jupyter notebook can Read more…

By pschatzmann, ago

Exchanging Data between MLlib, DL4J and Shogun

Each machine learning framework has its own basic data model and it is part of the ‘data preparation’ to convert the existing data to the required input format of the specific framework. Most of these frameworks also provide a direct access to some predefined data-sets (e.g iris, mnist etc) I tried to do a quick overview on how to exchange the data between the following frameworks DeepLearning for Java (DL4J) Scala MLlib Shogun so that Read more…

By pschatzmann, ago

Random Forrest Classifier in Spark ML

MLlib is Spark’s machine learning (ML) library. It’s goal is to make practical machine learning scalable and easy. I tried to make a complete step by step classification example using the Iris flower data set  using the BeakerX Jupyter kernel which covers the following steps Setup Data Preparation Testing and Prediction Validation The example is written in Scala but you could use any other language which is supported by the JVM. My example can be found the this GIST Read more…

By pschatzmann, ago