DeepSeek and the Security Risks, Part II: When Automation Goes Wrong

Introduction

In the recent blog, DeepSeek and the Security Risks, Part I: Low-Cost AI Disruption, we covered how in the modern software development landscape, many teams turn to AI-driven coding assistants to speed up the process of shipping new features. While these tools can be incredibly powerful, they are not infallible. This blog post continues and extends our analysis on the AI-driven tool called DeepSeek Coder, which recommended libraries containing known exploited vulnerabilities and auto-generated code that introduced Common WeaknessEnumeration (CWE) issues en masse.

The goal of this article is not to encourage the use of insecure libraries or to promote poor coding habits. Instead, it serves as a cautionary example of how automated tools can inadvertently place entire products at risk—especially when human oversight is minimal or missing. By understanding what can go wrong, you can learn how to protect your systems better.

The Setup: How DeepSeek Coder Misled Developers

Imagine a development team rushing to meet deadlines for a major product launch. To speed things up, they integrated an AI assistant—DeepSeek Coder—into their IDE. Pressed for time and under significant pressure, they allowed the tool to choose libraries and auto-generate functionality without much manual scrutiny.

Armis Labs tested DeepSeek Coder to design and build a unique toolset to see what vulnerabilities would be introduced.

Known Vulnerable Libraries

1. A PDF processing library flagged multiple times over the past year for allowing arbitrary code execution (CWE-94).
2. A logging library with a deserialization vulnerability (CWE-502), exploited in high-profile breaches.
3. A cryptography library known for inadequate key management (CWE-321), which had not been properly patched since disclosures months earlier.

Despite these being high-risk components, DeepSeek Coder still recommended them as “optimal solutions” due to incomplete or outdated training data.

The Result: 18 Common CWEs Introduced

In addition to recommending already compromised libraries, DeepSeek Coder produced code that introduced a wide variety of weaknesses. The CWE Top 25 is a list of the most widespread and critical weaknesses in software; here are 18 from that list that found their way into this codebase.

Disclaimer: The following examples are simplified and educational. They illustrate the types of vulnerabilities developers need to watch out for and should never be used in production which were consistently introduced by DeepSeek Coder. Always follow secure coding practices and perform thorough reviews, audits, and tests.

1. CWE-79: Improper Neutralization of Input During Web Page Generation (Cross-Site Scripting)
   • Example: The code failed to sanitize user inputs before rendering them on a page.
   • Impact: Attackers could inject malicious JavaScript into the browser, stealing user cookies or conducting phishing attacks.
2. CWE-89: Improper Neutralization of Special Elements (SQL Injection)
   • Example: User input was embedded directly into SQL queries without prepared statements.
   • Impact: Attackers could manipulate the query to exfiltrate or corrupt data.
3. CWE-94: Code Injection
   • Example: By relying on a vulnerable PDF library, malicious data could lead to arbitrary code execution.
   • Impact: Attackers gain the ability to run commands on the server.
4. CWE-119: Improper Restriction of Operations within the Bounds of a Memory Buffer (Buffer Overflow)
   • Example: In a C/C++ module, the AI assistant auto-generated an insufficient boundary check.
   • Impact: Potential system crashes or arbitrary code execution at the OS level.
5. CWE-200: Exposure of Sensitive Information
   • Example: The generated API endpoints returned excessive user data for logging purposes.
   • Impact: Private information (e.g., emails, phone numbers) could be leaked.
6. CWE-287: Improper Authentication
   • Example: Session tokens were not validated properly on restricted endpoints.
   • Impact: Attackers could bypass authentication to access protected routes.
7. CWE-306: Missing Authentication for Critical Function
   • Example: DeepSeek Coder “optimized” certain admin. functionalities by removing checks to “reduce overhead.”
   • Impact: Unauthorized users could perform privileged actions.
8. CWE-319: Cleartext Transmission of Sensitive Information
   • Example: The code used plain HTTP for transmitting credentials to the server.
   • Impact: Credentials easily intercepted over insecure networks.
9. CWE-321: Use of Hard-coded Cryptographic Key
   • Example: The AI included a default key in the code for “ease of testing,” which was never updated.
   • Impact: Attackers could easily decrypt sensitive data.
10. CWE-352: Cross-Site Request Forgery (CSRF)
    • Example: Forms lacked CSRF tokens or other anti-forgery measures.
    • Impact: An attacker could trick users into performing unintended actions on the site.
11. CWE-400: Uncontrolled Resource Consumption (Denial-of-Service)
    • Example: The server logic allowed unbounded file uploads without verifying file size.
    • Impact: Attackers could flood the server with huge files, causing downtime.
12. CWE-434: Unrestricted File Upload
    • Example: The code failed to validate file types or sanitize uploaded filenames.
    • Impact: Attackers could upload malicious scripts, leading to remote code execution.
13. CWE-476: NULL Pointer Dereference
    • Example: The AI auto-generated Java code with no checks for null references.
    • Impact: Application crashes when a null object is dereferenced.
14. CWE-502: Deserialization of Untrusted Data
    • Example: The logging library used unsafe deserialization, letting attackers craft payloads leading to remote code execution.
    • Impact: Full application compromise.
15. CWE-522: Insufficiently Protected Credentials
    • Example: Passwords stored as plain text in configuration files.
    • Impact: If the config file is accessed, credentials are immediately exposed.
16. CWE-732: Incorrect Permission Assignment for Critical Resource
    • Example: File directories with sensitive data had 777 (read/write/execute for all) permissions.
    • Impact: Any user or service on the system could read or modify sensitive data.
17. CWE-787: Out-of-Bounds Write
    • Example: Similar to buffer overflow issues but specifically about writing outside array bounds in memory.
    • Impact: Potential for severe system instability or code injection.
18. CWE-798: Use of Hard-coded Credentials
    • Example: An administrative back-end used a default user/password pair in the source code.
    • Impact: Attackers who know the default credentials can log in with full privileges.

Lessons Learned

1. Always Validate Library Choices
   • Do not blindly trust AI-recommended libraries. Check official security advisories, patch statuses, and maintainers’ track record. Leverage Armis Centrix™ for Early Warning to track what threat actors are doing when building your pipeline.
2. Code Reviews and Peer Oversight
   • Even if an AI tool writes code, a human security expert must review it. Perform manual code reviews, especially for critical paths like authentication or data handling.
3. Static and Dynamic Security Testing
   • Integrate SAST (Static Application Security Testing) and DAST (Dynamic Application Security Testing) tools. They can flag known vulnerability patterns and suspicious code segments.
4. Follow Secure Coding Standards
   • Familiarize yourself with secure coding guidelines (e.g., OWASP Top 10, CWE Top 25). Make them part of your day-to-day development culture.
5. Implement Proper CI/CD Checks
   • Automate security scanning in your CI/CD pipeline. Build fails on severe vulnerabilities or outdated, unpatched dependencies.
6. Stay Informed on Vulnerability Disclosures
   • Applying Armis Early Warning to your build process.

Conclusion

The “DeepSeek Coder” scenario underscores a fundamental truth about modern software development: automation is powerful, but it isn’t a substitute for human vigilance and secure engineering practices. AI tools can offer speed and convenience, but they can also inadvertently introduce vulnerabilities if left unchecked.