Senior Software Engineer (Backend)

Who We Are Back in 2019, spiderSilk was born with a bold idea: build regional, sovereign cybersecurity IP that could stand tall on the global stage. Our mission is to shake up the way organizations protect their ever-changing digital worlds — with continuous, intelligent, and autonomous security that doesn't miss a beat. We're a global mix of curious minds, problem-solvers, and passionate builders, all united by one goal: making the internet a safer place for everyone. Around here, we thrive on vision, energy, and a strong sense of ownership. If this feels like your kind of crew, you'll probably fit right in. About the Role As a Senior Software Engineer (Backend), you'll take a leadership role in designing and building the mission-critical systems that power spiderSilk's cybersecurity platform. You'll architect scalable, secure, and intelligent backend infrastructure — integrating ML pipelines, designing agentic AI solutions, and ensuring the platform can handle the demands of real-world threat intelligence at scale. You won't wait to be told what to do. You'll jump into architectural discussions, challenge assumptions, and drive decisions. You'll mentor junior engineers and set the bar for engineering quality across the team. What You'll Do • Architect and maintain distributed, microservices-based systems that operate reliably at scale • Own the design and delivery of standalone agentic AI solutions — from architecture through to production deployment • Build and optimise ML pipelines, agent workflows, and vector-based similarity search systems • Design and operate event-driven architectures using Kafka and message queues for real-time data streaming • Engineer and optimise complex database schemas across relational, graph, search, and analytical data stores • Lead data pipeline development for ingesting, processing, and surfacing threat intelligence data • Proactively contribute to architectural decisions — bringing opinions, trade-off analysis, and technical leadership without waiting to be asked • Mentor junior and mid-level engineers through code reviews, design discussions, and pair programming • Champion secure-by-design engineering practices across every layer of the stack Requirements Languages & StackMUST-HAVE • Python, Go, and Java — strong proficiency in all three with production-grade backend experience across each • Deep understanding of concurrency primitives — threads, goroutines, thread pools, locks, and race conditions; knowing when more parallelism makes things slower, not faster • Solid working knowledge of Docker and Kubernetes for containerisation and orchestration • Experience with Redis for caching, rate limiting, and distributed coordination Distributed Systems & ArchitectureMUST-HAVE • Deep experience designing and operating distributed systems and microservices architectures • Proven track record building event-driven systems (Kafka, RabbitMQ, or equivalent) • Expertise in data pipelines and stream processing at scale • Strong system design instincts — able to reason independently about trade-offs across reliability, performance, and security, and step into architectural decisions without prompting • Hands-on experience with wide-column stores (Cassandra or equivalent) for high-write, high-scale workloads • Working knowledge of object storage (S3 or compatible) and workflow orchestration tools (Dagster, Airflow, or similar) for managing complex data pipelines AI / Agentic SystemsMUST-HAVE • Hands-on experience building standalone agentic AI solutions in production — not just integrating APIs, but designing the agent architecture end-to-end • Deep understanding of vector databases and similarity search (e.g. Pinecone, Weaviate, pgvector, Qdrant) and how to apply them effectively in AI-powered pipelines • Proven work on agent optimisation and performance tuning — latency reduction, retrieval quality, context management, tool-call efficiency • Familiarity with agentic frameworks (LangChain, LlamaIndex, AutoGen, or similar) and the ability to go beyond them when needed • Practical experience with ML pipeline design, model serving, and feature engineering within production AI systems • Strong prompt engineering skills and up-to-date fluency with the evolving AI tooling ecosystem DatabasesEXPERT LEVEL • Expert-level skills in database schema design and optimisation across multiple paradigms • MySQL & PostgreSQL — advanced query optimisation, indexing strategies, performance tuning, and schema design for high-traffic systems • Elasticsearch — indexing, search relevance tuning, and cluster operations • Graph databases — production experience with Neo4j or equivalent (Amazon Neptune, ArangoDB); ability to model complex relationship data for threat intelligence or entity resolution • Working knowledge of analytical / OLAP databases (e.g. ClickHouse, BigQuery, Redshift, DuckDB) and when to apply them • Ability to select the right database technology for the problem and design schemas that scale Data Collection & Web Technologies • Experience collecting data from the web at scale — HTTP clients, headless browsers (Playwright or similar), handling sessions, rate limits, and sources that change without notice • Ability to parse large, malformed, or untrusted files and archives safely and efficiently at volume • Broad understanding of web servers, frameworks, CDNs and proxies — how they reveal themselves through headers, cookies, page content, and certificates • Familiarity with software fingerprinting and vulnerability mapping — understanding how product versions map to known vulnerabilities (CPE, CVE) Security • Working knowledge of cybersecurity principles and common security tooling • Demonstrated ability to apply secure coding practices and security-by-design thinking • Understanding of threat modeling, vulnerability patterns, and secure API design • Solid understanding of network security fundamentals — TCP/IP stack, DNS, TLS/SSL, HTTP/S, firewalls, and network traffic analysis • Familiarity with network attack patterns — DDoS, MITM, port scanning, packet sniffing, and lateral movement techniques • Experience working with or building systems that consume network telemetry data (e.g. NetFlow, PCAP, Zeek/Bro logs, SNMP) • Understanding of zero-trust architecture principles and how to apply them in distributed, internet-facing systems • Familiarity with security tooling such as Snort, Suricata, Wireshark, Nmap, or similar network inspection and monitoring tools DevOps & Operational Awareness • Basic working knowledge of CI/CD pipelines — setting up, maintaining, and debugging pipelines (GitHub Actions, GitLab CI, Jenkins, or equivalent) • Comfortable with deployment workflows — deploying services to VMs and Kubernetes clusters, managing rollouts, and handling rollbacks • Hands-on experience debugging live services on Kubernetes — inspecting pods, reading pod logs, exec-ing into containers, and triaging crashloops or resource issues • Ability to fetch and analyse logs from VMs — SSH access, journald/systemd logs, log rotation, and using tools like grep, awk, tail, and jq to debug production issues • Familiarity with observability tooling — working with log aggregation (ELK, Loki) and basic metrics/alerting (Prometheus, Grafana) to diagnose issues in production • Understanding of Kubernetes autoscaling in depth — resource requests/limits, HPA, and event-driven autoscaling (KEDA); able to diagnose OOM kills, CPU throttling, and unclean shutdowns Ownership & Leadership • 5–8 years of backend engineering experience with a proven track record of delivering at scale • Self-driven and proactive — comfortable jumping into architectural decisions, raising concerns early, and owning outcomes without hand-holding • Proven ability to lead projects and mentor engineers • Strong written and verbal communication across technical and non-technical stakeholders Nice to Have • Background in cybersecurity tooling, threat intelligence platforms, or security research • Exposure to cloud-native infrastructure (AWS, GCP, or Azure) • Experience with time-series databases (InfluxDB, TimescaleDB) relevant to security telemetry • Contributions to open-source AI or security tooling • Background in offensive security, penetration testing, or vulnerability research • Experience with threat intelligence work — tracking threat actors, leaked credentials, or underground forums • High-volume pattern matching or text processing — large regex sets or parsers working on untrusted, multilingual input at scale • Applied ML experience — clustering and visualising embeddings (e.g. UMAP) or other dimensionality reduction techniques for security data • Leading a project or initiative that spans multiple teams or services Why Join spiderSilk 🌐 Hard Problem, Real Impact Work on autonomous cybersecurity at global scale — a genuinely difficult engineering challenge with real-world stakes. 🏗️ Architecture From Day One Your voice shapes the system. You won't wait for permission — you'll be expected to drive decisions. 🤝 High-Ownership Team A tight-knit, globally distributed team building sovereign cybersecurity IP with full ownership of their work. 💡 Mission That Matters Competitive compensation, flexible working, and a mission to make the internet safer for everyone.