{"id":652,"date":"2018-11-21T23:40:16","date_gmt":"2018-11-21T22:40:16","guid":{"rendered":"https:\/\/www.pschatzmann.ch\/home\/?p=652"},"modified":"2020-11-21T22:22:50","modified_gmt":"2020-11-21T21:22:50","slug":"investor-lstm-stock-forecasting-harmonicstockoscillator","status":"publish","type":"post","link":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/","title":{"rendered":"Investor &#8211; Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator)"},"content":{"rendered":"<p><b>A Long short-term memory<\/b><span>\u00a0(<\/span><b>LSTM<\/b><span>) network is a special type of a\u00a0<\/span><a href=\"https:\/\/en.wikipedia.org\/wiki\/Recurrent_neural_network\" title=\"Recurrent neural network\">recurrent neural network<\/a><span>\u00a0(RNN).\u00a0It remembers values over arbitrary time intervals which makes it suited to be used for forecasting of time series.<\/span><\/p>\n<p>I was struggling with the <a href=\"https:\/\/deeplearning4j.org\/\">DL4J<\/a> implementation of the LSTM to be used to forecast stock data and I was not sure where the issue was:<\/p>\n<ul>\n<li>was the data not suited for forecasting ?.<\/li>\n<li>did I have a bug in my Iterator which was generating the 3d arrays ?<\/li>\n<li>did i have an issue with the scaling of the data ?<\/li>\n<li>was the network setup not correct ?<\/li>\n<\/ul>\n<p>So I decided to simplify the problem by using generated sinusoid data instead of real stock values in order to proof that the LSTM model is working as expected. Fortunately we can use the\u00a0<a href=\"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/dl4j-harmonicstockoscillator%C2%B6\/\">HarmonicStockOscillator<\/a> class to generate the test data.<\/p>\n<p>Here is my <a href=\"https:\/\/nbviewer.jupyter.org\/73753764f637b134e72730877f240355\">Gist<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A Long short-term memory\u00a0(LSTM) network is a special type of a\u00a0recurrent neural network\u00a0(RNN).\u00a0It remembers values over arbitrary time intervals which makes it suited to be used for forecasting of time series. I was struggling with the DL4J implementation of the LSTM to be used to forecast stock data and I [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":653,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[14,13],"tags":[],"class_list":["post-652","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","category-quantitative-trading"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Investor - Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator) - Phil Schatzmann<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Investor - Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator) - Phil Schatzmann\" \/>\n<meta property=\"og:description\" content=\"A Long short-term memory\u00a0(LSTM) network is a special type of a\u00a0recurrent neural network\u00a0(RNN).\u00a0It remembers values over arbitrary time intervals which makes it suited to be used for forecasting of time series. I was struggling with the DL4J implementation of the LSTM to be used to forecast stock data and I [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/\" \/>\n<meta property=\"og:site_name\" content=\"Phil Schatzmann\" \/>\n<meta property=\"article:published_time\" content=\"2018-11-21T22:40:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-11-21T21:22:50+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2018\/11\/forecast.png\" \/>\n\t<meta property=\"og:image:width\" content=\"650\" \/>\n\t<meta property=\"og:image:height\" content=\"492\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"pschatzmann\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"pschatzmann\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/\"},\"author\":{\"name\":\"pschatzmann\",\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/#\\\/schema\\\/person\\\/73a53638a4e34e8373405fd737dac9b1\"},\"headline\":\"Investor &#8211; Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator)\",\"datePublished\":\"2018-11-21T22:40:16+00:00\",\"dateModified\":\"2020-11-21T21:22:50+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/\"},\"wordCount\":156,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/#\\\/schema\\\/person\\\/73a53638a4e34e8373405fd737dac9b1\"},\"image\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.pschatzmann.ch\\\/wp-content\\\/uploads\\\/2018\\\/11\\\/forecast.png\",\"articleSection\":[\"Machine Learning\",\"Quantitative Trading\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/\",\"url\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/\",\"name\":\"Investor - Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator) - Phil Schatzmann\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.pschatzmann.ch\\\/wp-content\\\/uploads\\\/2018\\\/11\\\/forecast.png\",\"datePublished\":\"2018-11-21T22:40:16+00:00\",\"dateModified\":\"2020-11-21T21:22:50+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.pschatzmann.ch\\\/wp-content\\\/uploads\\\/2018\\\/11\\\/forecast.png\",\"contentUrl\":\"https:\\\/\\\/www.pschatzmann.ch\\\/wp-content\\\/uploads\\\/2018\\\/11\\\/forecast.png\",\"width\":650,\"height\":492},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/2018\\\/11\\\/21\\\/investor-lstm-stock-forecasting-harmonicstockoscillator\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Investor &#8211; Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/#website\",\"url\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/\",\"name\":\"Phil Schatzmann Consulting\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/#\\\/schema\\\/person\\\/73a53638a4e34e8373405fd737dac9b1\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/home\\\/#\\\/schema\\\/person\\\/73a53638a4e34e8373405fd737dac9b1\",\"name\":\"pschatzmann\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/wp-content\\\/uploads\\\/2022\\\/08\\\/pschatzmann.png\",\"url\":\"https:\\\/\\\/www.pschatzmann.ch\\\/wp-content\\\/uploads\\\/2022\\\/08\\\/pschatzmann.png\",\"contentUrl\":\"https:\\\/\\\/www.pschatzmann.ch\\\/wp-content\\\/uploads\\\/2022\\\/08\\\/pschatzmann.png\",\"width\":305,\"height\":305,\"caption\":\"pschatzmann\"},\"logo\":{\"@id\":\"https:\\\/\\\/www.pschatzmann.ch\\\/wp-content\\\/uploads\\\/2022\\\/08\\\/pschatzmann.png\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Investor - Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator) - Phil Schatzmann","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/","og_locale":"en_US","og_type":"article","og_title":"Investor - Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator) - Phil Schatzmann","og_description":"A Long short-term memory\u00a0(LSTM) network is a special type of a\u00a0recurrent neural network\u00a0(RNN).\u00a0It remembers values over arbitrary time intervals which makes it suited to be used for forecasting of time series. I was struggling with the DL4J implementation of the LSTM to be used to forecast stock data and I [&hellip;]","og_url":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/","og_site_name":"Phil Schatzmann","article_published_time":"2018-11-21T22:40:16+00:00","article_modified_time":"2020-11-21T21:22:50+00:00","og_image":[{"width":650,"height":492,"url":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2018\/11\/forecast.png","type":"image\/png"}],"author":"pschatzmann","twitter_card":"summary_large_image","twitter_misc":{"Written by":"pschatzmann","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/#article","isPartOf":{"@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/"},"author":{"name":"pschatzmann","@id":"https:\/\/www.pschatzmann.ch\/home\/#\/schema\/person\/73a53638a4e34e8373405fd737dac9b1"},"headline":"Investor &#8211; Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator)","datePublished":"2018-11-21T22:40:16+00:00","dateModified":"2020-11-21T21:22:50+00:00","mainEntityOfPage":{"@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/"},"wordCount":156,"commentCount":0,"publisher":{"@id":"https:\/\/www.pschatzmann.ch\/home\/#\/schema\/person\/73a53638a4e34e8373405fd737dac9b1"},"image":{"@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/#primaryimage"},"thumbnailUrl":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2018\/11\/forecast.png","articleSection":["Machine Learning","Quantitative Trading"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/","url":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/","name":"Investor - Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator) - Phil Schatzmann","isPartOf":{"@id":"https:\/\/www.pschatzmann.ch\/home\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/#primaryimage"},"image":{"@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/#primaryimage"},"thumbnailUrl":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2018\/11\/forecast.png","datePublished":"2018-11-21T22:40:16+00:00","dateModified":"2020-11-21T21:22:50+00:00","breadcrumb":{"@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/#primaryimage","url":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2018\/11\/forecast.png","contentUrl":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2018\/11\/forecast.png","width":650,"height":492},{"@type":"BreadcrumbList","@id":"https:\/\/www.pschatzmann.ch\/home\/2018\/11\/21\/investor-lstm-stock-forecasting-harmonicstockoscillator\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pschatzmann.ch\/home\/"},{"@type":"ListItem","position":2,"name":"Investor &#8211; Validating DL4J LSTM Stock Forecasting (using the HarmonicStockOscillator)"}]},{"@type":"WebSite","@id":"https:\/\/www.pschatzmann.ch\/home\/#website","url":"https:\/\/www.pschatzmann.ch\/home\/","name":"Phil Schatzmann Consulting","description":"","publisher":{"@id":"https:\/\/www.pschatzmann.ch\/home\/#\/schema\/person\/73a53638a4e34e8373405fd737dac9b1"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pschatzmann.ch\/home\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/www.pschatzmann.ch\/home\/#\/schema\/person\/73a53638a4e34e8373405fd737dac9b1","name":"pschatzmann","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2022\/08\/pschatzmann.png","url":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2022\/08\/pschatzmann.png","contentUrl":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2022\/08\/pschatzmann.png","width":305,"height":305,"caption":"pschatzmann"},"logo":{"@id":"https:\/\/www.pschatzmann.ch\/wp-content\/uploads\/2022\/08\/pschatzmann.png"}}]}},"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/posts\/652","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/comments?post=652"}],"version-history":[{"count":1,"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/posts\/652\/revisions"}],"predecessor-version":[{"id":2210,"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/posts\/652\/revisions\/2210"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/media\/653"}],"wp:attachment":[{"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/media?parent=652"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/categories?post=652"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pschatzmann.ch\/home\/wp-json\/wp\/v2\/tags?post=652"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}