I'm probably an idiot. Tell me I'm all wrong about this.
The danger is that quantum computers could factor large products well enough to reverse public keys, finding the associated private keys. Which would indeed be very bad. But this isn't quite a magic key that opens everything.
Public key crypto is used to set up a secure network connection, but it's not used to encrypt the data that flows on that connection. Quantum snooping would require an eavesdropper to intercept every bit on a connection, from initiation onward. And decrypting it would probably not be a real-time affair.
Public key crypto is also not used to protect your typical encrypted zip file or file system volume. Your Bitlocker and Veracrypt secrets aren't about to fall to quantum spies.
I'm bothered that so many popular articles about this issue draw no distinction between the classes of cryptography that are vulnerable and those that are not.
