The most surprising aspect of scaling transformer models for Sutskever is not any specific emergent capability, but rather that the models work and produce useful behavior at all, given his contrast with the pre-AlexNet era when neural networks produced nothing.