Check if the camera is lighter than the GoPro this gimbal is ment for. GoPros are usually a little heavier than other action cameras. The weight is very important for the gimbal in order to have the expected momentum. Espacially if the camera is lighter, the gimbal will start oscillating. Try adding small weights and check what happens.
Xiaomi yi 2k which i have now weight 72 g the same as gopro 3, and i have that problem, and im waiting for my New xiaomi yi 4k plus that weight 102g. One more question, is that buzzing causing any problems? F.e. Burn motor or sth else?
I would still try adding some weight. Since you replaced the original case by a 3D-printed one I guess you might have taken a few grams away. It is better to get rid of the buzzing. If it is not too intense it will not cause any problems.
I have idea, i made two of them one 3d printed and one from plastic that i bend, plastic is better than 3d printed and i think i have done mount holes for bearing and motor in bad places, So when i Will make New i Will try to balance it with camera before drilling mounting holes So it should work.