Garry Jenkins, 39, David Powell, 50, Phillip Hill, 45, and Charles Breslin, 62 died when the mine shaft flooded at Gleision Colliery last week. Three miners managed to escape.
The Wales Office will oversee the investigation which will draw on technical expertise from the HSE. It said that HSE specialist mine inspectors are on site and working closely with South Wales police, who are leading the investigation.
The Welsh Office said that at a later point it may be deemed appropriate for primacy of the investigation to be passed to HSE, as has happened in other incidents but it is too early to say now whether this will happen.
The Welsh office said at this stage it is too early to state possible causes and has urged the media to “refrain from speculation”.