Author: ecadmin

This example shows how to get a list of the top gainers on the day in US stocks using the Interactive Brokers (IBKR) API for Python.

The API works by connecting to a locally running Trader Workstation (TWS) instance. Make sure TWS is up and running; both live trading a paper trading logins will work.

In TWS, we need to enable EClients in TWS under Global Configuration / API. This will allow clients from localhost to connect to TWS over HTTP.

Install the IBKR API package using:

$ pip install ibapi

We extend both EClient and EWrapper to inherit the functionality of reading the scanner feed.
We call the scannerData() method of the superclass to get the actual data. We can ignore most of the parameters to scannerData(), but need to include them to match the signature.

Note that we need to launch a listener in a thread in parallel to the main script.
We connect to the data feed, read some items and disconnect.

The following script will get the first ten of the top percentage gainers at the time it is run:

import time
import threading

from ibapi.client import EClient
from ibapi.scanner import ScannerSubscription
from ibapi.wrapper import EWrapper

# Class extending IBKR scanner base classes.
class IBapi(EWrapper, EClient):
  def __init__(self):
    EClient.__init__(self, self)

  def scannerData(self, reqId, rank, contractDetails, distance, benchmark, projection, legsStr):
    super().scannerData(reqId, rank, contractDetails, distance, benchmark, projection, legsStr)
    position = rank + 1 # Zero based indexing.
    print(f"{position}: {contractDetails.contract.symbol}")

  def scannerDataEnd(self, reqId):
    print("End of scanner data.")
    self.disconnect()

# Function to run continuously.
def ibApiRunLoop(ibApiApp):
  ibApiApp.run()

# Main script.

TWS_HOST = '127.0.0.1'
TWS_PORT = 7497
CLIENT_ID = 12345

ibApi = IBapi()
ibApi.connect(TWS_HOST, TWS_PORT, CLIENT_ID)

thread = threading.Thread(target=ibApiRunLoop, args=(ibApi,), daemon=True)
thread.start()

time.sleep(1) # Allow time for connection.

scannerSub = ScannerSubscription()
scannerSub.numberOfRows = 10
scannerSub.instrument='STK' # Stocks.
scannerSub.locationCode='STK.US.MAJOR' # US major exchanges.
scannerSub.scanCode='TOP_PERC_GAIN' # Top percentage gainers.

requestId = 1000 # An arbitrary request ID.
filters = [] # No special filters needed.

ibApi.reqScannerSubscription(requestId, scannerSub, filters, filters)

time.sleep(5) # Allow time to receive data.

ibApi.disconnect()

Example output:

1: LUCY
2: TNMG
3: DEVS
4: SRXH
5: SMCX
6: SMCL
7: LOBO
8: LGMK
9: HIMZ
10: SMCI
End of scanner data.

Note that to get more details, e.g. the last price of the tickers, we need to query data specific to each ticker.

References

https://www.interactivebrokers.com/campus/ibkr-quant-news/interactive-brokers-python-api-native-a-step-by-step-guide/

https://www.interactivebrokers.com/campus/ibkr-quant-news/implementing-market-scanners-using-tws-api-part-ii/

 

The following example shows how to retrieve the value of the float of a stock in Python using the Yahoo Finance (yfinance) library.

Note that there is no value returned specifically for Free Float, but Float Size should be equal or close enough for many purposes.

The following script retrieves the float size for any ticker symbol provided as input.

The float is under the Key Statistics (keyStats).

import yfinance as yf

ticker = input("Ticker symbol: ")

stock = yf.Ticker(ticker)

keyStats = stock.info

floatShares = keyStats.get("floatShares")

print("Float size: " + str(floatShares))

Example run:

Ticker symbol: AREB
Float size: 1026592

 

The following is a simple example showing how to scrape a web page in Python using the BeautifulSoup library.

The example web page we use is Hacker News:

https://news.ycombinator.com/

The goal of the example is to print just the headlines from the page.

Scraping and parsing a web page requires first understanding the HTML document structure of the target page.
In this case, the relevant portion of the HTML structure is roughly as follows:

<table>
 <tbody>
  <tr>
   ...
   <td>
    <span class="titleline">
     <a href="https://...">Headline</a>
    </span>
   </td>
   ...

Crucially, BeautifulSoup can find elements by many different criteria, regardless of depth of nesting.

One of the most useful techniques is searching by CSS class. In this case, we need to find anchor elements inside spans with the class “titleline”, inside table cells.

First, we create a BeautifulSoup object and use the find methods to drill down into the HTML elements in the DOM.

Note that findAll() finds a list of elements, while find() gets a single element.

The entire script is below.

scrape-example.py

from bs4 import BeautifulSoup
from urllib import request

url = "https://news.ycombinator.com/"

htmlPage = request.urlopen(url)

soup = BeautifulSoup(htmlPage, "html.parser")

# Find list of elements by CSS class.
spanList = soup.findAll(
  "span", 
  attrs={"class": "titleline"}
)

for span in spanList:
  anchor = span.find("a")
  headline = anchor.text
  print(headline)

Running the script:

$ python scrape-example.py

Example output of headlines:

The subtle art of designing physical controls for cars
WASM will replace containers
Backblaze Drive Stats for 2024

...

 

Tool Use for LLMs is the idea of giving a language model the ability to take actions by executing external code.

LangChain is a framework which uses Chain-of-Thought (COT) prompting in order to generate steps for a plan of action and then actually carry out those steps.

A tool is defined as software functionality which can be triggered through matching a Tool class, which is associated with a function. The function code specifies the exact tool functionality.
In the simplest case the return value from a function reachable through a tool is a string.

In this example we use a local LLM running through Ollama: Llama 3.
One simple tool is provided: a function that gets the current time. Normally, an LLM will say it does not have access to a clock when asked the time in a prompt. This tool gives the model that ability.

Overall, the code does the following:

• Define a function that returns the current time
• Define time_tool as a Tool class with a good description of what the tool does and links to the time function

Note that matching the description is probabilistic and not 100% guaranteed. Hence, we make the description as complete and explicit as possible to increase the likelihood of matching. Note also that some models are better than others; LlaMa 3 performs quite well.

• We link the LLM and the tools list together using LangChain’s initialize_agent()
• Run a prompt asking for the current time. The model answering will trigger the use of the time tool, and the return value is used in the final result

The output is set to verbose, so we can see the actions planned and executed.

Before running the code, make sure Ollama is running and Llama 3 is available with:

$ ollama run llama3

Make sure the LangChain libraries required are installed:

$ pip install langchain

$ pip install langchain-community

$ pip install langchain-ollama

Note: at the time of writing using LangChain >= 0.3.x works well.
A specific version install may be required, for example:

$ pip install langchain==0.3.14

Here is the complete code:

from datetime import datetime
from langchain.agents import initialize_agent, AgentType
from langchain.tools import Tool
from langchain_community.chat_models import ChatOllama


def get_time(query: str):
  # So we can see when the tool code is reached:
  print("get_time() called.")
  return "The time is: " + datetime.now().strftime("%H:%M:%S") + "."

time_tool = Tool(
  name="TimeTool",
  description="Fetch time information. Able to get the current time.",
  func=get_time
)

tools = [time_tool]

llm = ChatOllama(model="llama3")

agent = initialize_agent(
  tools,
  llm,
  agent_type=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
  verbose=True,
  handle_parsing_errors=True
)

query = "What is the time?"

response = agent.invoke(query)

print(response)

Note that the output will not necessarily be consistent between runs.
The following is an example run:

> Entering new AgentExecutor chain...
Let's get started!

Question: What is the time?

Thought: Let's get started!

Thought: Since I need to find out what the current time is, 
I should use the TimeTool.

Action: TimeTool

Action Input: None needed for this tool, it just provides 
the current time.
get_time() called.

Observation: The time is: 22:42:02.

Thought:Final Answer: The time is: 22:41:59.

> Finished chain.
{'input': 'What is the time?', 
 'output': 'The time is: 22:41:59.'}

 

A factorion is a number which equals the sum of the factorials of each of its digits.

For example: 145 is a factorion, because:

1! + 4! + 5! = 1 + 24 + 120 = 145

The following Python code will determine if a given number is a factorion.

import math

def isFactorion(n):
  digits = getDigits(n)

  digitSum = 0

  for digit in digits:
    digitSum += math.factorial(digit)

  return n == digitSum


# Helper function.
# Get a list of digits as integers for a number n.
def getDigits(n): 
  return [int(digitChar) for digitChar in list(str(n))]


# Examples.
if isFactorion(145):
  print("145 is a factorion")
else:
  print("145 is not a factorion")

if isFactorion(185):
  print("185 is a factorion")
else:
  print("185 is not a factorion")

Output for the two examples:

145 is a factorion
185 is not a factorion

 

Suppose we want to generate a range of characters, similar to how the range() function returns a list of numbers between a given range.
This can be easily achieved with special constants in the string module.

To get the range between ‘a’ and ‘z’, we can use string.ascii_lowercase.

For example:

import string

for char in string.ascii_lowercase:
  print(char)

Output:

a
b
c
...
z

To generate the range between ‘A’ and ‘Z’, we can use string.ascii_uppercase.

For example:

import string

for char in string.ascii_uppercase:
  print(char)

Output:

A
B
C
...
Z

The constants can be used like any list, e.g. for iteration or testing membership of an element.

 

The following example shows how to calculate Exponential Moving Average (EMA) values for a stock, using various periods, in Python.

We are using the TA (Technical Analysis) library.

The example below calculates the 10 EMA on a one minute chart and the 25 EMA on a 1 minute chart.

The window parameter can be adjusted to any desired value to calculate other useful periods such as the 50 or 200 EMA.

The example uses Yahoo Finance data via the yfinance library.

calculate-emas.py:

import yfinance as yf

from ta.trend import EMAIndicator

ticker = 'META'

EMA10_WINDOW = 10

EMA25_WINDOW = 25

# EMAs on 1 minute timeframe.
data = yf.download(tickers=ticker, 
   period='1d', 
   interval='1m'
)

closeValues = data['Close']

emaIndicator10 = EMAIndicator(close=closeValues, 
   window=EMA10_WINDOW
)

emaIndicator25 = EMAIndicator(close=closeValues, 
   window=EMA25_WINDOW
)

# These return Pandas series.
emaSeries10 = emaIndicator10.ema_indicator()

emaSeries25 = emaIndicator25.ema_indicator()

print('Last 5 values for EMA 10: ')
print(emaSeries10.tail(5))

print()

print('Last 5 values for EMA 25: ')
print(emaSeries25.tail(5))

Example run:

$ python calculate-emas.py

[*********************100%%**********************] 
1 of 1 completed

Last 5 values for EMA 10:
Datetime
2024-11-04 14:57:00-05:00 562.326530
2024-11-04 14:58:00-05:00 562.321704
2024-11-04 14:59:00-05:00 562.249574
2024-11-04 15:00:00-05:00 562.289524
2024-11-04 15:01:00-05:00 562.278698
Name: ema_10, dtype: float64

Last 5 values for EMA 25:
Datetime
2024-11-04 14:57:00-05:00 562.883210
2024-11-04 14:58:00-05:00 562.838347
2024-11-04 14:59:00-05:00 562.768088
2024-11-04 15:00:00-05:00 562.745104
2024-11-04 15:01:00-05:00 562.705480
Name: ema_25, dtype: float64

References

https://technical-analysis-library-in-python.readthedocs.io/en/latest/ta.html

Using Google AdSense, we may encounter issues with an ad serving limit placed on the account due to invalid traffic concerns.

There are many spam bots and automated scripts running on the Internet which may contribute to website traffic, but this traffic does not represent legitimate users.

Specifically to WordPress sites, the first step to eliminate the invalid traffic is to install plugins to block spam and click fraud. Two good options are given below.

For preventing click fraud:

ClickCease Click Fraud Protection

For preventing comment spam:

Antispam Bee

Further, one of the sources of the invalid traffic is automated hacking attempts from various scripts.

Bots scanning websites are attempting to run exploits, e.g. against login pages.

Therefore, we see many requests in the server logs with attempts to reach:

/wp-admin/login/

/wp-admin/login/login.php

and so on.

If the login page URL is completely unknown, it helps to stop these kinds of requests: if the usual default login URL returns a 404 a script will likely not try as many further malicious requests.

We want to make it difficult to predict the actual valid URL.

First, generate a UUID (Universally Unique ID) for the login page using:

Online UUID Generator

The custom login URL can be, for example:

{some-site.com}/{UUID}_custom_login

Because the probability of guessing a specific UUID is very low, it should be very difficult to reach the valid custom login URL from any external script.

To change the login URL use the plugin:

Change wp-admin login

This adds new settings in WordPress. Look under:

Settings / Permalinks / Change wp-admin login

Enter the customized URL here and click Save Changes.

 

When attempting to renew an SSL Certificate for a domain using Certbot on Ubuntu, we may encounter the following problem:

Renewing an existing certificate for <www.my-domain.com>
Performing the following challenges:
http-01 challenge for <www.my-domain.com>
Waiting for verification...
Challenge failed for domain <www.my-domain.com>

This may mean that the domain cannot be accessed properly, i.e. the test requires an external connection to our server to succeed over TLS (SSL).
We need to make sure that the domain can be reached on port 443.

Thus, ensure the firewall (UFW) allows connections on port 443:

$ ufw allow 443

Depending on the user executing the command, this may need to be run as superuser:

$ sudo ufw allow 443

We can now try the Certbot renew command again.

 

We can use the Python Technical Analysis Library (ta) for the RSI (Relative Strength Index) calculation.

A simple and free API for finance data is Yahoo Finance (yfinance); the most convenient way to call it is using the yfinance Python library.

First, install the Technical Analysis library:

$ pip install ta

Install the Yahoo Finance library:

$ pip install yfinance

We use NVDA as an example, on a 5 minute timeframe.

The following script will retrieve the data and pass the results to the TA library to calculate the RSI.
The results are stored in a Pandas series; after retrieval we print out a sample of the latest values.

import yfinance as yf

from ta.momentum import RSIIndicator

ticker = 'NVDA'

# Make sure to use a window with enough data for the RSI calculation.
data = yf.download(tickers=ticker, period='5d', interval='5m')

closeValues = data['Close']

# Use the common 14 period setting.
rsi_14 = RSIIndicator(close=closeValues, window=14)

# This returns a Pandas series.
rsiSeries = rsi_14.rsi()

# Latest 10 values of the day for demonstration.
print(rsiSeries.tail(10))

Below is the output for a single run. The series contains timestamps and RSI values.

Datetime
2024-05-24 12:35:00-04:00    52.733213
2024-05-24 12:40:00-04:00    56.698742
2024-05-24 12:45:00-04:00    56.991273
2024-05-24 12:50:00-04:00    62.709294
2024-05-24 12:55:00-04:00    55.964462
2024-05-24 13:00:00-04:00    56.631537
2024-05-24 13:05:00-04:00    51.238671
2024-05-24 13:10:00-04:00    53.740376
2024-05-24 13:15:00-04:00    51.827860
2024-05-24 13:20:00-04:00    52.300520
Name: rsi, dtype: float64