Almost all low-tax ('offshore') jurisdictions control banking activity, not least in order to prevent money laundering.
Domestic and international banking are usually separately licensed; domestic banking is sometimes reserved for local banks.
Offshore banks are normally free of taxation - but there is usually a hefty annual license fee; and the process of 'harmonization' that took place in the first decade of the 21st century has led in some cases to the imposition of a low tax rate across all sectors.
Banking secrecy is usually very good in offshore jurisdictions, often enforced by statute; but jurisdictions increasingly have international agreements that permit breaches of confidentiality in some circumstances, often under the auspices of a Tax Information Exchange Agreement (TIEA). TIEAs became prevalent between 2008 and 2010, when the OECD succeeded in imposing standardized rules for mutual information exchange on almost all countries and territories. Typically, a TIEA (or equivalent wording in a Double Tax Treaty) will permit one country to extract information from another when it has prima facie evidence of wrongdoing (including tax evasion) on the part of a named person or company. It is early days in the process, but the high-tax countries are doggedly pursuing their 'anti-offshore' agenda, and the long-term survival of banking secrecy, even in its last bastions such as Switzerland and Andorra, seems doubtful.