Your dog's "mode" determines its behavior. It is located above the top right corner of the dog's name box in the training GUI.
The dog will peacefully follow along behind you and won't attack anything. In this mode, you can play fetch with the dog bone.
The dog will wander around the area, staying close to its food bowl until you command it to do something. You have to be sure that your dog is "set" to the food bowl by bringing it within one block of the bowl.
The dog will follow you and attack anything that is attacking you. It will also attack anything that you attack first.
The dog will follow you, but will immediately attack any mob that gets too close.
The dog will follow you, but only attack a mob when you use the command emblem on it.