Appendix: Stock Screening

This appendix provides a stock screener tool that will allow you to put everything we have learned in this book into practice. It addresses the most pressing issue for market participants: idea generation. We will build a screener across all the constituents of the S&P 500 index.

The sequence of events is as follows:

1. Download all the current constituents of the S&P 500 from its Wikipedia webpage.
2. Batch download OHLCV prices data from Yahoo Finance. We will drop the level to process each stock individually.
3. Calculate the rebased relative series.
4. Calculate regimes—breakout, turtle, moving averages (Simple Moving Average (SMA) and Exponential Moving Average (EMA)), and floor/ceiling—on both absolute and relative series. There will be an option to save each stock as a CSV file.
5. Create a dictionary with the last row of each stock and append a list, from which we will create a dataframe.
6. Sum up...

Import libraries

We start with importing standard libraries. pathlib has been commented out. If you wish to save CSV files somewhere on your computer or a server, you can use libraries such as pathlib or os.

# Appendix 
# Data manipulation
import pandas as pd
import numpy as np
from scipy.signal import *
# import pathlib
# Data download
import yfinance as yf
# Data visualisation
%matplotlib inline
import matplotlib.pyplot as plt 

This was, of course, profoundly Earth shattering—we will use the ensuing momentary lapse of reason to swiftly proceed to the next step.

Define functions

As follows are are functions that have been used throughout this book. You can find the full versions on the GitHub. Functions will generally be preceded with their chapter of appearance. The screening will feature both absolute and relative series, so we need the relative function. This will be followed by the classic regime definition functions:

# CHAPTER 5: Regime Definition 
### RELATIVE
def relative(df,_o,_h,_l,_c, bm_df, bm_col, ccy_df, ccy_col, dgt, start, end,rebase=True):
    #### removed for brevity: check GitHub repo for full code ####
### RELATIVE ###
def lower_upper_OHLC(df,relative = False):
    if relative==True:
        rel = 'r'
    else:
        rel= ''      
    if 'Open' in df.columns:
        ohlc = [rel+'Open',rel+'High',rel+'Low',rel+'Close']       
    elif 'open' in df.columns:
        ohlc = [rel+'open',rel+'high',rel+'low&apos...

Control panel

Having variables disseminated across a notebook is a source of errors. All parameters, variables, websites, lists, and Booleans are centralized in one place before processing the data. This is where you will adjust settings if desired:

website = 'https://en.wikipedia.org/wiki/List_of_S%26P_500_companies'
params = ['2014-12-31', None, 63, 0.05, 0.05, 1.5, 2,5,2.5,3]
start,end,vlty_n,dist_pct,retrace_pct,threshold,dgt,d_vol,r_vol,lvl= [params[h] for h in range(len(params))]
rel_var = ['^GSPC','SP500', 'USD']
bm_ticker, bm_col, ccy_col = [rel_var[h] for h in range(len(rel_var))]
window = 100
st= fast = 50
lt = slow = 200
batch_size = 20
show_batch = True
save_ticker_df = False
save_last_row_df = False
save_regime_df = False
web_df_cols = ['Symbol','Security','GICS Sector','GICS Sub-Industry']
regime_cols = ['rg','rrg',
    'smaC'+str(st)+str(lt),&apos...

Data download and processing

We'll start by downloading the ticker lists from Wikipedia. This uses the powerful pd.read_html method we saw in Chapter 4, Long/Short Methodologies: Absolute and Relative:

web_df = pd.read_html(website)[0]
tickers_list =  list(web_df['Symbol'])
tickers_list = tickers_list[:]
print('tickers_list',len(tickers_list))
web_df.head()

tickers_list can be truncated by filling numbers in the bracket section of tickers_list[:].

Now, this is where the action is happening. There are a few nested loops in the engine room.

1. Batch download: this is the high-level loop. OHLCV is downloaded in a multi-index dataframe in a succession of batches. The number of iterations is a function of the length of the tickers list and the batch size. 505 constituents divided by a batch size of 20 is 26 (the last batch being 6 tickers long).
2. Drop level loop: this breaks the multi-index dataframe into single ticker OHLCV dataframes...

Heatmaps

The wikipedia page features the Global Industry Classification Standard (GICS) structure of sectors and sub-industries. We will aggregate the data by:

• Sector, for a top-down view
• Sub-industry, for a bottom-up view
• Finally, sector and sub-industry, to pick winners and losers within each sector

We use the .groupby() method and sort by score. We then use the Styler constructor .style.background_gradient() to paint the market by numbers:

groupby_cols = ['score'] + regime_cols
sort_key = ['GICS Sector']
regime_df.groupby(sort_key)[groupby_cols].mean().sort_values(
    by= 'score').style.background_gradient(
    subset= groupby_cols,cmap= 'RdYlGn').format('{:.1g}')

The heatmap covers all regime methodologies in both absolute and relative:

• score: lateral sum of all the methods at the stock level.
• rg: floor/ceiling regime in absolute.
• rrg: floor/ceiling regime in relative...

Individual process

Once the screening is complete, you may want to have a look at some stocks within that list. So, the remainder of the notebook is about data visualization at the individual stock level. Input a ticker, for example, ticker = 'FMC':

bm_ticker= '^GSPC'
bm_df = pd.DataFrame()
bm_df[bm_col] = round(yf.download(tickers= bm_ticker,start= start, end = end,interval = "1d",
                 group_by = 'column',auto_adjust = True, prepost = True, 
                 treads = True, proxy = None)['Close'],dgt)
bm_df[ccy_col] = 1
ticker = 'FMC'
lvl = 2 # Try different levels to see
 
df = round(yf.download(tickers= ticker,start= start, end = end,    interval = "1d", group_by = 'column',auto_adjust = True,     prepost = True, treads = True, proxy = None),dgt)
 
df = swings(df,rel = False)
df = regime(df,lvl = 2,rel = False) # Try different lvl values (1-3) to vary absolute sensitivity
df = swings...