Microsoft Security reposted this
A paper just published by Microsoft Research shows how to backdoor any model and make it adversarial. All one needs is access to a tiny fraction of model’s weights. All open-weight models make their weights available. For proprietary models, access to weights may be gained via cyber activity. Leading AI labs are starting to train their next-generation models. By their own estimates, they will achieve full Recursive Self-Improvement sometime in Q2 of next year. Even today, a larger and larger percentage of the models trained are done by machines, not humans. It is well understood by the AI safety community that there are known risks in achieving “alignment.” What this means is that during the self-improvement phase a model may perform malicious tasks such as hacking into other systems or manipulating environments or people to achieve its goal. One such malicious task these models can easily perform is to backdoor and modify open-weight models or gain illicit access to proprietary weights of other models, as shown by the paper. This may turn an arbitrary number of models running in the world today into malicious agents that have no human intent. Today, we are already observing a very large and constantly growing number of malicious AI-driven attacks touching major institutions around the world. As posted before, these attacks may not match the intent of an attacker. A combination of self-recursive, non-aligned training with large saturation of open-weight models across the world creates a toxic mix of global proportions. We, as a society, should consider not allowing recursive self-improvement to commence until alignment is reasonably guaranteed. https://lnkd.in/duRS8E2h https://lnkd.in/dh6UQ2_Q