Allow external access to the HDFS cluster

[nightkr]

Currently we just expose overlay network addresses in both hdfs-site.xml and dynamic registrations to the namenodes. This prevents users from accessing HDFS from outside of the k8s overlay network.

This requires:

• Namenodes must be accessed using stable addresses (such as a LoadBalancer Service where available)
  • The generated discovery profile must be altered to use this address
• Datanodes must be accessed using their associated NodePort Service addresses

Datanode addresses cannot currently be overridden in HDFS, I have outlined the changes required for this in https://github.com/stackabletech/hadoop/tree/spike/override-datanode-id. This is a purely server-side change.

I have a spike that implements this semi-manually in https://github.com/stackabletech/hdfs-operator/tree/spike/external-access, if we consider that access pattern acceptable then we can start looking into how to integrate it better into both HDFS and the operator.

[soenkeliebau]

I have created a sister issue to this one #235 which we can use to track the upstream work to properly fix this.
For this issue I think we should simply shoot for patching the version of HDFS that we ship, as timelines around upstream work can be completely unpredictable.