It's a design bug. Since X1 was supposed to connected at all times, when they removed that feature due to complaints, they left something in the architecture of the system that even if you have the physical disc to install from, it actually will download the game from the server like a digital instead. That's why it takes so long, b/c you aren't really copying from the disc to install, you are downloading on your internet. Couple that with it trying to d/l and install updates while it d/l the game at the same time.
Just select "Go offline" before you insert the game to install and it will install quickly from the disc like normal.