Items with hundreds of children can adversely affect performance and usability.
In some cases, you can take a hybrid approach, where you access some data directly in the external system, but maintain additional data in Sitecore. For example, you could implement a custom data template field to allow users to select a record in the external system, and then create items in the CMS containing fields of that type to store information about those records that the external system cannot store.
If you decide not to use data providers or the hybrid approach, then you can use .NET presentation components such as sublayouts and web controls to access the external system through web services or .NET APIs provided by that system or that you write to expose that system.
John West, Sitecore CTO: Whether to implement a custom data provider in the Sitecore CMS