Hack/Reduce Ottawa: Shopping & Dating Hacks

We had an excellent Hack/Reduce event again this weekend in Ottawa. Our knowledge has been expanded again, we now know how desperate daters can get and at what time people do their online shopping among other things.

In order to get all the tech to work and keep the clusters humming a lot of coffee and pizza was consumed. I would be afraid to calculate how much.

Most importantly, we saw a lot of great Hack/Reductions by the participants. Actually, first and foremost it was a great weekend for the Hack/Reduce team again – the best way to spend a Saturday – for real! Thanks!

We want to thank all the participants who made it out and the sponsors of course: Hopper, Infoglutton and Shopify.

Here’s a short description of some of the presentations we saw:


Jean-Claude Batista (@jcbatista) , Taswar Bhati (@taswarbhatti) , Joel Sachs, Andrew Clunis (@orospakr), (Petro Verkhogliad)

The Hackify team worked on the shopify dataset with information about US Shopify orders. The team used Ruby and the streaming api.

Firstly, the team analyzed where people shop the most. Naturally, the answer was California (largest state).
Second, the team wanted to find the date when people had shopped the most, which was on August 19th between noon and 6pm ($185891).

The team concluded that using Ruby with the streaming api makes it easy to do map/reduce and that Hadoop is cool!


Petro Verkhogliad (@vpetro)

Petro first worked with the Hackify team but then turned to using Python which he was more familiar with. Petro also analyzed the Shopify shopping data.

Petro found that people shop the most at 9am in the morning and 9pm in the evening. The most popular shopping day is Friday with Saturday as a close second.

Geographically most shopping is done by Californians ordering from California.


Rod Dunne, Marc Lepage, Mohamed Mansour (@mohamedmansour) , Alexis Brunet

Code: https://github.com/mlepage/HackReduce

Team-RIM was voted as winner by the participants. They worked on the google n-grams data and the Amazon review data. First, the team calculated the amount of alliterative 2-grams by letter per year. They also calculated the amount of alliterative 2-grams by letter per year.

For the amazon review data, team-RIM calculated the average rating given on specific dates calculated over all of the years in the dataset. You can see that all products are almost given a rating of 4. You can also notice that right after christmas the ratings drop off, ie. people give worse review right after christmas.

The amazon dataset also includes data over how useful reviews are. The reviews can be voted up or down on amazon by users. Team-RIM analyzed this data and came to the conclusion that reviews that give a higher rating to a product are considered more useful.
The team also analyzed the usefulness of reviews based on the review length. From the picture we can see that 50-60 character reviews are considered most useful. As a bonus, the team calculated that products received worse reviews as time went by.

Pascal – Analysis of the most desperate daters

Pascal from Hopper wanted to find the most desperate daters. He analyzed the amount of profile views by users. According to his analysis, some users are checking so many profiles that with a 30-second timeframe it amounts to full-time work. (~17k views per month… Which amounts to something like 60 profile views every working hour…)

Pascal also found the most visited profiles.

Lastly, Pascal analyzed how users use the Mate1 site. The results were quite surprising, as 23% of users only view one other profile. 65% only ever check 10 profiles.


JF (@jeanfrancoisim) from Hopper also analyzed the dating dataset.

First, JF mapped the birth year of users to their hotness by gender. JF calculated hotness with the following formula: (msgs received+msgs sent) * ((msgs received+10)/(msgs sent+10)).

The red dots represent women and blue dots men. We can clearly see that the woman demographic is all in all younger. I’m afraid to draw any other conclusions from the results…

Next JF mapped hotness and height by gender. You could easily see that men are taller than women but it is unclear if height directly influences hotness.

As a last analysis JF mapped income to hotness. Most people answered “rather not say” to this question, why most datapoints are in the second column. It’s also unclear if income influences hotness.


Steven Noble (@snoble), Chris Saunders(@chris_saunders) , David Underwood (@davefp)

The XKCD team wanted to test the internet “truism” that has been said in a xkcd comic: “Wikipedia trivia: if you take any article, click on the first link in the article text not in parentheses or italics, and then repeat, you will eventually end up at “Philosophy”

The team didn’t really end up with a result for this since it turned out to be difficult to only include the actual article content and not all of the other links on the page. They started their work in Clojure but later ended up using java instead.

Team Dating 8

Martin Samson (@pgdown) , Edgar Acosta, David Germain

Team 8 explored the dating dataset with the goal of measuring popularity/hotness and correlating it to other factors.
The team started by analyzing some basic measures for profiles.

They created their own popularity formula.

The team made some nice graphs mapping times listed vs. messages received, times listed vs. profile views, popularity vs. number of times viewed..

The team used streaming with Python and fought with it for a while. Lesson: Do not use the same file name for the mapper and the reducer scripts.

Learnt: Do not use the same file name for the mapper and reducer scripts.

Team weather and crime

Richard Desmarais, Chris Camden, Philippe Savoie, Ryan McLeod, Eric Ax, Manuel Belmadani (@pragmatwit)

The weather and crime team created a web interface that could answer queries such as “What is the maximum temperature in january 2007”. When the search query is launched it will run through the dataset and give you the answer. The team used python streaming.

Team UOttawaNLP + others

Chris Fournier (@cfournie ), Oana Frunza, Alistair Kennedy, Russell Luo, Dominic Plouffe (@dplouffe )

Team UOttawa analyzed the sentiment of the Amazon reviews. As expected, there were clear differences in sentiment between the good and the bad reviews.


In the end, we let the participants vote for the best Hack/Reduction. Team-RIM with their n-grams and Amazon analyses took the win… Congrats!


Thanks to everyone for coming, the Hack/Reduce team had a great time and it was amazing to meet you all. We hope you keep hacking and we hope we’ll see you next time!


Enhanced by Zemanta


  1. Trackback: review site
  2. Trackback: Rhinoplasty los angeles
  3. Trackback: Arizona Divorce Attorney
  4. Trackback: more information
  5. Trackback: study in florence italy
  6. Trackback: http://www.youtube.com/watch?v=m8ZhGO6NXUE
  7. Trackback: amazon.com
  8. Trackback: vancouver bc injury attorney
  9. Trackback: VEMMA
  10. Trackback: professional photo studio
  11. Trackback: How To Start a Conversation With a Girl
  12. Trackback: 15m hdmi cable
  13. Trackback: check this out
  14. Trackback: site
  15. Trackback: sixpack trainingsplan
  16. Trackback: Calgary SEO
  17. Trackback: Flipora
  18. Trackback: www.3dingconsulting.com
  19. Trackback: Dove posso comprare silos per lo stoccaggio dei mangimi?
  20. Trackback: find out more
  21. Trackback: Consigli su progettazione elettronica nella zona di Cuneo
  22. Trackback: go
  23. Trackback: movers dc
  24. Trackback: rivertrees residences
  25. Trackback: Subway Surfers - Play free Subway Surfers game online
  26. Trackback: phone speaker
  27. Trackback: true tech new blog
  28. Trackback: hamptonbaylight.com
  29. Trackback: hampton bays
  30. Trackback: file share
  31. Trackback: happywheels demo game
  32. Trackback: www.happywheelsdemogameonline.com
  33. Trackback: happywheelsdemo2
  34. Trackback: http://babybridalshowergames.com
  35. Trackback: qwxgvnmkfbrvecganfhv
  36. Trackback: axmcsnrcaxmgcnacgnr
  37. Trackback: garcinia cambogia maximum daily dosage
  38. Trackback: scnkgrfmstngjsngmgcrthv
  39. Trackback: how to increase testosterone
  40. Trackback: check this site out Scottsdale DUI attorney
  41. Trackback: svsjgvgvbbvcfncggjkdf
  42. Trackback: pemutih badan
  43. Trackback: simulatie lening
  44. Trackback: acgggggggdbjmhkfasdj
  45. Trackback: which is the real garcinia cambogia
  46. Trackback: dr oz garcinia cambogia side effects
  47. Trackback: www.tiremart30.com
  48. Trackback: Anonymous
  49. Trackback: ahorrador de combustible
  50. Trackback: Window Cleaning Supplies
  51. Trackback: ejemplos de valores familiares
  52. Trackback: cirugia plastica new york
  53. Trackback: paintless dent repair training
  54. Trackback: seguros en el salvador
  55. Trackback: paintless dent removal training
  56. Trackback: ถ่ายรูป
  57. Trackback: real estate in singapore market
  58. Trackback: garcinia cambogia side effect
  59. Trackback: The Glades Condo Floorplan
  60. Trackback: pdr-training.net
  61. Trackback: paintless dent removal tools
  62. Trackback: http://www.nodents.com
  63. Trackback: paintless dent repair training
  64. Trackback: add my business
  65. Trackback: home remodeling Atlanta Ga
  66. Trackback: Atlanta contractors
  67. Trackback: handyman atlanta
  68. Trackback: que son los gobiernos
  69. Trackback: commonwealth towers floor plan
  70. Trackback: dr oz 2 week rapid weight loss shopping list
  71. Trackback: dr oz 2 week diet
  72. Trackback: gxcrcfgrtgsgabdjnhacfg
  73. Trackback: รับปริญญา
  74. Trackback: how to get slotomania free coins
  75. Trackback: vapor e cigarettes
  76. Trackback: http://www.newpropertyout.blogspot.com
  77. Trackback: sdhfdjscnrgsbrnfgssgsxgc
  78. Trackback: get his heart
  79. Trackback: check out now
  80. Trackback: deer antler velvet
  81. Trackback: www.newpropertyout.blogspot.com
  82. Trackback: dr oz and garcinia cambogia
  83. Trackback: visit this site
  84. Trackback: gcnksdmfaxamrfngsemrfgs
  85. Trackback: garcinia cambogia reviews dr oz
  86. Trackback: dr oz and garcinia cambogia
  87. Trackback: garcinia cambogia free trial and detox
  88. Trackback: hcg diet drops
  89. Trackback: testosterone booster reviews
  90. Trackback: http://deer-antlerspray.net/
  91. Trackback: insanity workout sale
  92. Trackback: rachael ray diet garcinia
  93. Trackback: dr oz fat burners for women
  94. Trackback: pure garcinia cambogia extract
  95. Trackback: how do you get rid of bed bugs
  96. Trackback: koszenie trawy
  97. Trackback: diani beach hotels
  98. Trackback: 3 credit scores
  99. Trackback: labour day pictures
  100. Trackback: Celebrity Images