Some time back, while working on a project for my Computational Finance course, I needed stock quotes for prominent firms from the BSE stock exchange. I needed data for the last five years, and manually copying the data from online sources was going to be very tiresome. While Google Finance, Yahoo Finance and other websites let you export search results for a data range into a CSV (comma separated values) file, doing this for 20 stocks was going be to a boring and repetitive task. Then I realized that for Yahoo Finance, by manipulating the URL pattern I could request the stock quote for any stock and for any date range. Once I realized this, it was a matter of writing a small program in Ruby to automate the entire task.
First of all, I used a text file with CSV data containing the stock symbol and the date range for the which data was to be downloaded. The format followed was STOCK-SYMBOL, FROM DATE, TO DATE. All dates were in the format dd-mm-yyyy. Here is a sample text file:
MOSERBAER.NS,1-5-2002,31-10-2008 HDFCBANK.NS,1-5-2002,31-10-2008 |
The program that I wrote is shown below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | require 'open-uri' require 'csv' input_file = CSV.open('ticker_codes.txt', 'r', ',') yahoo_url = 'http://ichart.finance.yahoo.com/table.csv?s={SYM}&d={t_m}&e={t_d}&f={t_y}&g=d&a={f_m}&b={f_d}&c={f_y}&ignore=.csv' input_file.each do |row| ticker = row[0] from_day, from_month,from_year = row[1].split(/-/) to_day, to_month,to_year = row[2].split(/-/) url = yahoo_url.gsub('{SYM}',ticker) url = url.gsub('{f_m}',from_month) url = url.gsub('{f_d}',from_day) url = url.gsub('{f_y}',from_year) url = url.gsub('{t_m}',to_month) url = url.gsub('{t_d}',to_day) url = url.gsub('{t_y}',to_year) aStock = File.new(ticker+'.csv','w') aStockCsv = open(url) aStockCsv.each do |line| aStock<<line end aStock.close end |
Basically what I did was read the CSV file. For each line of the file, I separated the stock symbol, from date and to date. The dates were further split into month, day and year. Then using pattern replacement, I manipulated the URL to have all the parameters set as Yahoo Finance needed them to be. Then it was just a matter of reading this URL and writing the CSV file represented by the URL to disk.
End result – I got a nice folder of CSV files, one for each stock, and ready for input into Excel, Matlab or any other data analysis tool.