01/13/2024: Generate Obsidan Page From Browser Bookmarks
Goal
To gain a searchable set of pages that represent the bookmarks that I have collected over the years.
Introduction
I had over 100 bookmarks and had no idea what most of them were. I haven’t looked at them, ever. But I do use Obsidian to organize research topics and write. I thought it would be useful if I could turn those unused bookmarks into searchable page.
Each page would include at least a summary and categories.
This script was created for my own use. Therefore there is not a lot of documentation. If you need help with anything, please create an issue.
The script ignores URLS that contain: [‘docs.google’, ‘reddit’, ‘slack’, ‘twitter’]. Each one would require a different approach and I did not need them. If you do, create an issue to discuss your needs or create a pull request with your approach.
GitHub Link
https://github.com/medined/bookish-fortnight - see example Obsidan pages here.
Ollama
I wanted a process that ran locally so that no cost was incurred. In order to enable this, I used Ollama and the solar LLM. I tried other large language models but solar
seemed to produce the best results.
You need to have Ollama installed locally as well as runnning inside the container. The Ollama inside the container is a server responding to the code’s API calls.
Docker
I used a container to run Ollama. Since I already had docker installed, this seemed like the best approach.
Process
See the readme in the project.
06/12/2021: Fetching Stock Symbols
Goal
To get a list of stock symbols for the NYSE and NASDAQ markets.
Introduction
I’ve been writing python programs to help plan my investment trades. The basis of everything is to know what stocks are available. The code below gets that information in a Pandas dataframe from an FTP site.
Code
This is the NYSEMarket.py file.
import pandas as pd
# See https://www.nasdaqtrader.com/trader.aspx?id=symboldirdefs#nasdaq for information.
class NYSEMarket:
url = "ftp://ftp.nasdaqtrader.com/symboldirectory/otherlisted.txt"
original_columns = [
'ACT Symbol',
'Security Name',
'Exchange',
'CQS Symbol',
'ETF',
'Round Lot Size',
'Test Issue',
'NASDAQ Symbol'
]
# These columns are ignorable after they are used for filtering.
ignorable_columns = [
'CQS Symbol',
'ETF',
'Exchange',
'NASDAQ Symbol',
'Round Lot Size',
'Test Issue',
]
def __init__(self):
self.df = pd.read_csv(self.url, delimiter='|')
self.df = self.df[self.df['Test Issue'] == 'N']
self.df = self.df[self.df['ETF'] == 'N']
self.df.drop(self.ignorable_columns, axis=1, inplace=True)
self.df.rename(columns={'ACT Symbol':'Symbol'}, inplace=True)
self.df = self.df[:-1]
This is the NASDAQMarket.py file.
import pandas as pd
# See https://www.nasdaqtrader.com/trader.aspx?id=symboldirdefs#nasdaq for information.
class NASDAQMarket:
url = "ftp://ftp.nasdaqtrader.com/symboldirectory/nasdaqlisted.txt"
original_columns = [
'Symbol',
'Security Name',
'Market Category',
'Test Issue',
'Financial Status',
'Round Lot Size',
'ETF',
'NextShares',
]
# These columns are ignorable after they are used for filtering.
ignorable_columns = [
'ETF',
'Financial Status',
'Market Category',
'NextShares',
'Round Lot Size',
'Test Issue',
]
def __init__(self):
self.df = pd.read_csv(self.url, delimiter='|')
self.df = self.df[self.df['Test Issue'] == 'N']
self.df = self.df[self.df['ETF'] == 'N']
self.df.drop(self.ignorable_columns, axis=1, inplace=True)
self.df = self.df[:-1] # drop the last row.
This is how I use those classes:
#!/usr/bin/env python
from NASDAQMarket import NASDAQMarket
from NYSEMarket import NYSEMarket
import pandas as pd
nasdaq_market = NASDAQMarket()
nyse_market = NYSEMarket()
df = pd.concat([nasdaq_market.df, nyse_market.df])
df.rename(columns={'Symbol': 'symbol'}, inplace=True)
df.rename(columns={'Security Name': 'company_name'}, inplace=True)
df = df[~df.symbol.str.contains('\.')]
df = df[~df.symbol.str.contains('\$')]
df = df[~df.company_name.str.endswith('- Rights')]
df = df[~df.company_name.str.endswith('- Subunit')]
df = df[~df.company_name.str.endswith('- Subunits')]
df = df[~df.company_name.str.endswith('- Trust Preferred Securities')]
df = df[~df.company_name.str.endswith('- Unit')]
df = df[~df.company_name.str.endswith('- Units')]
df = df[~df.company_name.str.endswith('- Warrant')]
df = df[~df.company_name.str.endswith('- Warrants')]
df.sort_values(by=['symbol'], inplace=True)
print(df.head())
The output looks like this:
symbol company_name
0 A Agilent Technologies, Inc. Common Stock
1 AA Alcoa Corporation Common Stock
4 AAC Ares Acquisition Corporation Class A Ordinary ...
0 AACG ATA Creativity Global - American Depositary Sh...
1 AACQ Artius Acquisition Inc. - Class A Common Stock