quick question. Can ALL files use template override?

[budfox]

I don't want to edit core files, but the override guide says it only works where 'custom' already exists