gparrella/web-app repository overview

1. Why forecast stock indexes?⁠

2. How to Run⁠

3. Enabling Technologies⁠

4. Dataset⁠

5. Learning Strategies⁠

6. Models⁠

7. Automated Hyperparameters Tuning⁠

8. Results⁠

9. Credits⁠

⁠Why Forecast Stock Indexes?

Given the inherent risk, uncertainty, and dynamic nature of stock markets, accurately forecasting stock indexes can be important for investors, analysts, and businesses as it can help them make informed decisions about the markets and plan for the future.

In particular, we chose stock market indexes from the MAMAA group, an acronym that encompasses five of the top performing technology stocks in recent history:

• META
• AMAZON
• MICROSOFT
• APPLE
• ALPHABET

Hence, this project aims to predict the future values of these stock indexes using machine learning techniques.

⁠How to Run

Download the web app image running the following command

  docker pull gparrella/web-app:latest

⁠Deploy on Kubernetes Cluster

Open a terminal in WebApp folder and run the following command

  kubectl apply -f webapp_deployment.yaml

Verify that the web app entrypoint is active running the following command

  kubectl get services

Verify that the deployment is in the READY 2/2 state running the following command

  kubectl get deployments

Now you can access to the web app' s homepage at the following address

  http://localhost:30001

If you want to remove the app from Kubernetes, open a terminal in WebApp folder and run the following command

  kubectl delete -f webapp_deployment.yaml

If you want to remove the docker image too, run the command:

  docker image rm gparrella/web-app --force

⁠Deploy on Docker Container

Open a terminal in WebApp folder and run the following command

  docker run -itd --name web-app-test --publish 8501:8501 gparrella/web-app

Now you can access to the web app' s homepage at the following address

  http://localhost:8501

If you want to remove the docker container, you can open a terminal and run the following command:

  docker rm web-app-test --force

If you want to remove the docker image too, run the command:

  docker image rm gparrella/web-app --force

⁠Streamlit share

The web app is also available at this link⁠ with Streamlit share free hosting.

⁠Enabling Technologies

• Kubeflow, used for creating pipelines for training models
• Jupyter Notebook, used for codes the pipeline's components
• Kale & Katib, Kubeflow Add-Ons used to automate the tuning of hyperparameters of models
• Docker, used to create the container in which the web app and its dependencies are placed
• Kubernetes, used to deploy the containerized web app
• Python, used in the entire development life cycle.

⁠Dataset

The data that were used to do model training were obtained from Yahoo! Finance's API. They include, for each index, data from the time the index was listed on the stock exchange. In particular, the yfinance⁠ library was used.

Specifically, index values at the close of the trading day (Close value) were taken, in that the objective here is to develop a model to forecast the Close value of the next trading day. For the training of the model only the last 10-15 years of data were used, so that the models are not biased on data associated with economic situations other than current ones.

⁠Learning Strategies

Two learning strategies were adopted in order to compare them and choose the best one for the problem at hand.

The first strategy involves creating specific models, that is, trained only on the dataset in the index on which they want to specialize.

The second strategy, on the other hand, involves creating general models, that is, trained on a single dataset that contains data from all indexes.

⁠Models

The models used in this project are:

• Stacked Long Short-Term Memory (LSTM) network, a type of recurrent neural network that are particularly well-suited for time series forecasting.
• Automatic Relevance Determination (ARD) Regressor, a type of machine learning models that are used for regression tasks. It has proved to be very effective for forecasting.

⁠Automated Hyperparameters Tuning

To do automatic tuning of model hyperparameters is Kale's Katib functionality. Specifically, Katib automates the tuning based on parameters that are specified by the user. The main parameters are as follows:

• For each hyper-parameter to be tested, the range of values to be tested;

• The objective metric to be used to determine the optimal values of the hyperparameters (e.g. MAE in our case), as well as its optimisation (e.g. minimisation in our case);

• The search algorithm to be used during optimisation (e.g. Bayesian Optimisation in this case);

• The number N of experiments to be performed (e.g. N=20 in our case);

Once the configuration phase is complete, the search algorithm will create the N experiments, each of which will run the pipeline we defined to do the training of the models, using the different values of the hyperparameters specified previously and selected according to its policies.

At the end of an experiment, it is possible to view the metric value obtained, as well as the combination of hyperparameters that optimises that metric.

⁠Results

The following results represent the Mean Absolute Error (MAE) calculated on the test set. The unit of the MAE is USD.

⁠Credits

This project was developed by Ernesto Mancusi⁠ and Giuseppe Parrella⁠

⁠License

This code is distributed under the GNU General Public License v3.0 License.