Tips and Troubleshooting
If you want a more stock Android experience, you can install a custom launcher, which will let you hide all the Amazon apps you don't need. I like Nova Launcher, but there are many others available.
Assuming you are done installing the APKs from unknown sources, it's a good idea to go back and turn that option off. It's in Settings > Security & Privacy. Toggle off the Apps from Unknown Sources option. It'll prevent you from accidentally downloading malicious software.
If you run into problems installing apps from the Play Store, try clearing the app data. Head to Settings > Apps & Notifications > Manage All Applications. Look through that list and find the Play Store. Tap to edit the settings. The first step is to tap Force Stop to shut down the Play Store. Next, tap Storage, then Clear Data (or Clear Storage, depending on which version of Fire OS you're running). Once that's done, restart your device and try logging in to the Play Store again.