There is no real abstraction layer at all. All you have to do is ensure the correctness of endianess with the appropriate calls and everything else is exactly the same. Most of the consumer ARM CPUs are little endien anyhow. Most of the issues I've encountered are usually sloppy driver writing such as taking x86 hardware based caching ops for granted and not call the appropriate cache flush/invalidate functions when operating on other ARCHs that do those things in software for DMAs.