Vote Charlie!

Mint batch transaction import hack

This was posted four years ago when I was 29 years old. Hopefully I have matured since then!

Posted at age 29.

Years ago, I started using Mint to aggragate my financial information. The idea made sense, like browser based Gmail made sense compared to using Outlook on the desktop. Mint seemed to be the first free service to connect with most financial institutions. Now there are many such services, and Mint has not managed to resolve any of the usability issues I have experienced all those years. I was about to look for alternatives today till I found a workaround for my latest annoyance: lack of bulk import.

Mint does not integrate with 1st Financial Bank USA, apparently due to the latter’s reluctance or inability to provide secure access to transaction data. As a result, I almost never use the card, but I started to a few months ago. I therefore need to enter all those transactions into Mint manually. Before resigning to my fate, I had a look at the request my browser made to post a transaction manually. It turned out I could edit this request and resubmit it by command line, allowing me to automate adding my transactions.

Procedure

Before I submitted a transaction to Mint using Firefox, I opened the network tools by selecting Tools > Web Developer > Network or pressing Option+Command+Q. Then I submitted the following transaction.

manual-transaction.png

manual-transaction.png

In the network tools request list, I looked for POST requests and saw one to updateTransaction.xevent and clicked on it. Then I clicked the Params tab to see what data was sent. It was clearly the transaction.

post-request.png

post-request.png

I right clicked the updateTransaction.xevent request and selected Copy as cURL.

I right clicked the <code>updateTransaction.xevent</code> request and selected <em>Copy as cURL</em>.

I right clicked the updateTransaction.xevent request and selected Copy as cURL.

This gave me the following long string, which I broke up into lines for each parameter here for presentation. Note I also removed the cookie information and an account number and token.

curl 'https://mint.intuit.com/updateTransaction.xevent' \
--2.0 \
-H 'Host: mint.intuit.com' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0' \
-H 'Accept: */*' \
-H 'Accept-Language: en-US,en;q=0.5' \
--compressed \
-H 'X-Requested-With: XMLHttpRequest' \
-H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' \
-H 'Referer: https://mint.intuit.com/transaction.event' \
-H 'Cookie: ...' \
-H 'DNT: 1' \
-H 'Connection: keep-alive' \
--data \
'cashTxnType=on&'\
'mtCashSplit=on&'\
'mtCheckNo=&'\
'tag2079584=2&'\
'tag896243=0&'\
'tag896244=0&'\
'tag896245=0&'\
'task=txnadd&'\
'txnId=%3A0&'\
'mtType=cash&'\
'mtAccount=...&'\
'symbol=&'\
'note=THIS%20IS%20A%20NOTE&'\
'isInvestment=false&'\
'catId=2&'\
'category=Shopping&'\
'merchant=TEST%20TEST%2FTEST%20TEST%40TEST&'\
'date=06%2F07%2F2017&'\
'amount=10000.01&'\
'mtIsExpense=true&'\
'mtCashSplitPref=1&'\
'token=...'

To post new transactions to Mint, I needed to simply edit the merchant, date and amount items in the string following the last option of the curl command, the --data option. Also, there is apparently a bug in Firefox that results in an erroneous --2.0 in the curl command. It seems this might have been intended to be --http2, but that did not work for me. Removing the --2.0 worked fine.

I created a file 1fb.csv that contained a list of my transactions with the date, amount and merchant separated by a tab (\t) like this:

02/13	10.81	AMAZON MKTPLACE PMTS AMZN.COM/BILL WA
02/13	103.28	AMAZON MKTPLACE PMTS AMZN.COM/BILL WA
02/14	236.07	FOSTER SMITH MAIL ORDR 800-381-7179 WI

Then I wrote a short shell script into which I pasted the copied cURL command in place of <PASTE CURL COMMAND HERE> before making a few edits:

#!/usr/bin/env bash

while read p; do
  IFS=$'\t' read -ra COLS <<< "$p"
  tdate="${COLS[0]}/2017"
  tdate="${tdate//$'/'/%2F}"
  amount="${COLS[1]}"
  merchant="$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "${COLS[2]}")"
  printf "%q " 'merchant='$merchant'&date='$tdate'&amount='$amount
  <PASTE CURL COMMAND HERE> && echo
done <1fb.csv

I removed the --2.0 as mentioned and then replaced the merchant, date and amount with the variables such that ...merchant=...&date=...&amount=...& became merchant='$merchant'&date='$tdate'&amount='$amount'&.

I saved that as mint.sh alongside the CSV and made it executable and ran it: chmod +x mint.sh; ./mint.sh

For each success, cURL output {"task":"txnAdd","mtType":"CASH"}. To my delight, all the transactions showed up in Mint, accessible by the 1FB tag I set up for these transactions and used for my initial manual add.

Join the movement!

Show your support by signing up to be among the lucky few notified when I manage to blog. This could be as often as once a day or as infrequently as yearly!

  • Kestrel

    Thank you so much! I can confirm this method still works. I was able to import 2800ish transactions over a few runs with out to many issues. Some notes that may help others.

    1.) I used chrome to copy the cURL (bash) command. When I got it I had to change --data-raw to just --data. This may be from my version of cURL being older.
    2.) I was able to get about 1700 transactions imported before the session expired. The session as of Jan 2021 is 20min.
    3.) Sort your transactions by date so you can easily find where they failed. You can then adjust your spreadsheet. Make a new manual import to copy the curl command and run it all again.
    4.)There is no way to do a bulk remove with in Mint. I made a test account to practice with.
    5.) I created a custom tag in Mint called "Mass Import" and used it when I did my manual transaction that I got my cURL command from. This way I can filter my transactions to show just the transactions from the mass import.
    6.) Date codes need to be MM/DD/YYYY. You will need to update the bash script to remove /2017 from the date section.
    7.) I had to separate my expenses and my incomes. I made one manual transaction with income selected and a second with expenses selected and ended up with two different cURL commands and ran them separately.

  • Thank you very much... Still working in July 2020.

    Here is a C# version for those like me whose scripting chops aren't up to stuff:
    https://gist.github.com/brinko99/b8e3df3a97a8aec113e1385e6cbc976b

  • Dee

    Sadly, I couldn't get this to work. I can't tell if it's because POST elements have changed somewhat, or that there was a syntax error I couldn't spot. I also don't know bash well enough to reformat the date beyond a hard-coded 2020. If anyone has success with this let me know.

  • Barringer

    I just got it to work with hardcoded year (of 2019, as that was the year of the transactions I was adding). The format of the dates in the file was 11/26. What did you see? Just an echo of the three fields?

  • Brad

    Would anyone mind dumbing this down even further for someone who uses Windows 10? I know nothing about shell scripts...but Optum bank has decided to block Mint, and it would sure be nice to upload transactions into Mint instead of doing it manually.

blog comments powered by Disqus