I have read plenty of literature in this context (see links below). The presence of classic vs new Azure management portal, Roles vs App Services, Azure Website vs Azure Web Apps only complicates the matter. Although I believe I have good understanding of these terms and the latters are more or less improved re-incarnations of the formers in the above comparisons, the literature is written mostly using the former of these and not necessarily depicts the options available in the new Management portal.
So what I am really looking forward to understand very clearly is:
RE:Can I RDP to Azure App Service Web App underlying compute resource?
No, you are not allow to RDP into the machine. You application is running within a sandbox, depending on your plan, one machine might running multiple sandbox from different user.
RE: Can I install custom third party software components (e.g. Crystal Reports) to Azure App Service Web App underlying compute resource?
No, the machine will upgrade periodically, content might be reset. Also user that app is running on will not have the privileged to install application. If your application is depending on other dependencies, e.g asp.net web application, you should specify all dependencies during build time (nuget).
Check the first two reference you provide, they are talking about Azure Virtual Machine, not Azure App Service, where Azure Virtual Machine is SaaS, Azure App Service is a PaaS. If you are looking for something in the middle, please try Azure Cloud Service, it supports RDP and I believe you should be able to install third party apps with on start script.