Gradient descent is quite amazing in its power and simplicity. If we can figure out a good loss function and a model architecture with enough parameters, calculus can basically take care of the rest.
This pretty much my same pick. I'm completely blown away by the possibilities of a combination of gradient descent and clever loss/architectures. After learning how output shape and loss definition can result in wildly different tasks such as landmark detection, classification, or single shot multibox detection(bounding boxes AND classification wuuut?!) it really gets the imagination running wild.
