I seem to remember a tie being sold during the Coronation in 1953 and the shop being told to withdraw it. I do not know whether it was using banned images or whether it was an unauthorised item.
I know you can't use the image of the House of Commons portcullis.
You could find an image of a crown on free image websites (clipart) , surely, where no copyright applies. But if you do use the crown and/or orb, just don't wear it to Buckingham Palace
The problem you may face is that it may be claimed that your use of arms may give the impression of royal connection and is calculated to deceive, you should check the trade marks act 1994 and obtain permission from the Lord Chamberlain office before proceeding.