Nerd Thoughts

This is a non-academic yet academic page. The primary purpose is fun. However, the topics I write about are quite meaningful (to me, and my extremely nerdy mind). I do not claim that anything here is scientifically correct, just some thoughts I wonder about. I also would love it to receive answers, opinions, and/or other ideas to include. Don't hesitate to shoot me an email!

Discriminator vs. generator

May 14, 2017: One of my favorite recent papers is the GAN paper by (Goodfellow et al., NIPS, 2014). Mainly because in their acknowledgments they thank a brewery for stimulating their creativity (which I found purely hilarious and awesome). But also the concept of a discriminator and generator playing a min max game for training is very interesting. Actually, one of my colleagues always critiques my ideas and so I told him "sometimes I feel you and I are involved in a min max game like in GANs. I try to generate ideas and results, but you discriminate them. You are the max to my min". Ever since, I often see min max games in my daily life: my qual exam with the lovely committee I had, the peer review process (specially when there's a rebuttal, we get an extra SGD iteration), preparing a paper draft with my advisor, having arguments with friends in general, etc...

The paper will be accepted with probability p

May 14, 2017: So recently my first ICML paper was accepted. That was amazing. But few days ago, I was discussing with some colleagues about our chances of making it based on how the reviews looked. Someone said "I think there is a 70% chance" to which I replied "Nah, I would say 40%". A friend of mine said "You can't really assign a probability, what does it mean after the decision notification?" That was a legit question to which I replied "It's like quantum mechanics, I cannot say for sure if the paper is accepted or not, but each state has a probability, and yes after measurement the state collapses." That got me thinking, a lot of our life events could be modeled like that, we often say "oh there's a high chance this or that will happen" and eventually it does happen or not. When we talk like that, are we doing quantum mechanics without knowing it?

Training at the gym

May 14, 2017: One day I went to the gym and thought "I'm doing unsupervised training". We were just done with a ML paper submission in our group so the following day I told my advisor about it. He said "well, make sure your model does not overfit". That was a good point, ever since, I have been applying dropout (in the Geoff Hinton sense, if you didn't get it, congrats, you are a normal person, not a nerd). Unfortunately, looks like I'm always stuck in a local minimum (or maximum for that matter).

Dropout was used by hippies before it was cool

May 14, 2017: Some time ago I met a couple of neuroscientists from Berkeley. Naturally, they were telling me about how cooler Berkeley is compared to Stanford. I said I heard it is a very hippy place, and of course, they were proudly telling about the consumption of neuron-weakening substances there (if you didn't get this, congrats, heaven waits for your pure heart). Talking to these neuroscientists about those substances I said: "I just realize something, when someone smokes, it's like they are applying dropout to their brain". To my surprise they loved this joke and one of them started faking the smoking process saying "hold on, I am generalizing..." Actually, that was a lot of fun, but we always hear of artists (painters, poets, musicians, etc...) producing their best works when high. Isn't that exactly dropout??