BO 4.0 is completely new 64 bit architecture. So go through the documentation of BO 4.0 setup and sizing.
After installation, do complete testing before going live in PRODUCTION. This process will take time.
As long as you follow best practices, you should be good. But still we never know. Depending on your current environment, you may see some issues. Search in SCN for all the documentation.
Go through the 3.1 SP6 release notes, see if any of your issues fixed what you are seeing now in SP4.
But your ultimate goal will be moving to BO 4.1 version.