Many of us would have heard loud hums, crackles, and pops, when sound gear is being connected together (which could be CD players, guitars, keyboards, amplifiers, etc). While turning the input volume down before doing so stops you hearing the noises, it's still possible to damage input and output stages when high voltages get involved. The best thing to do is disconnect the power lead from the device you're attaching first, then make your audio lead connections, and finally connect the power lead last. Not only does it prevent such noises, it also prevents electrical damage from happening. However, once there is a ground connection between one thing and the other (e.g. via one of the audio leads, or a dedicated ground lead between them), other patch leads can usually be changed around without serious problems.
Equipment that is mains-powered often uses a switch-mode power supply, these days, and it's output is often floating at half the AC supply voltage (which is much stronger than any audio signals by a very large amount), rather than held at ground potential (zero volts), especially if it only has a two-pin mains plug (though many devices with a grounded three-pin mains connector will still have this problem). Many keyboards, mixers, amplifiers, etc, are powered this way. It's to do with how their power supply is designed, and the usual approach that they use to filter noise getting into or out of the mains side of the supply (with two capacitors going from the live and neutral AC connections to the common rail of the low voltage supply inside their power supply).
Even non-mains powered devices (such as acoustic guitars with an amplified pick-up fitted inside them, and some effects pedals) will often suffer from loud crackles and pops when patching and un-patching them. Though this is for different reasons—it's is usually due to them running their battery voltage through the barrel of the jack. Inserting a guitar lead completes the circuit between battery and pre-amp, switching it on. And disconnecting the lead automatically disconnects the battery. A better design, which rarely seems to be done, is to use a socket with a separate switching circuit (one that's mechanically operated by the insertion of a jack, but not electrically connected to it). About the only way to deal with this noisy problem is to mute the mixer or amplifier as the leads are connected, and keeping all the connections in a good and clean condition (so you don't get loud crackles when moving around).
A common way to deal with connecting mains-powered musical instruments is to use a DI (direct inject) box between instrument and mixer. It allows high-level line sources to be connected to mike inputs (if the DI box has attenuator features). And it can have a ground-lift feature to break ground loops between interconnected equipment (which should be the only [*] approach taken for solving ground loop problems, never remove the earth pins from three-pin mains connectors). If both sides of the connection are earthed, then lifting the grounding from the audio signal path between them can remove some hum and noises. But if the instrument is not grounded via its mains power lead, then the audio ground usually needs to be connected between the devices.