Unless things have changed, one of the key benefits of DNNs was that you trained them layer by layer.

You also want to be able to train the DNN on your unlabelled data and the SVM on your much smaller labelled set.