I had more or less the same issue too.
as previously said, custom ctrl is not a good way (to me) as user can disable by error a field and all is lot. I prefer to clean at Submit.
go to Advice on clearing down data on hiding a Group - #3 by lyzergic for seing the way to do . I update the post with my own little corrections