Interview Q&A
Fifty-five short questions with crisp answers, grouped by theme, and eight scenario prompts with outlines of what a strong answer covers: a crashing pod, a slow pipeline, zero-downtime database changes, a latency spike, a pipeline design for twenty services, a full disk, a dangerous Terraform plan and 502s after a deploy.
What Interviewers Look For
DevOps interviews test three things: whether you know the fundamentals (how Linux, containers, Kubernetes, pipelines and infrastructure code actually work), whether you can debug (a systematic method under pressure), and whether you can design (trade-offs, not buzzwords). This page has 55 short questions grouped by theme, then eight scenario questions with outlines of strong answers.
| Level of answer | What it sounds like |
|---|---|
| Weak | Names a tool ("I would use Prometheus") without saying what question it answers |
| Good | Explains the mechanism and the trade-off, and names a concrete symptom and fix |
| Strong | Starts with the goal and constraints, proposes a method, says what could go wrong and how they would verify the result |
- 1. Stop the bleeding if users are affected (roll back, scale, fail over, flag off). 2. Scope it: who, what, since when, what changed. 3. Form a hypothesis from signals (metrics, logs, traces), not from guesses. 4. Test one thing at a time. 5. Fix, verify with the same signal that revealed it, and write down the follow-up. Saying this out loud is half the answer.
Each theme below links to the page that teaches it in depth: 01 Linux & Shell Essentials, 02 Docker, 03 Kubernetes Core, 04 Kubernetes Operations, 05 CI/CD Pipelines, 06 Infrastructure as Code, 07 Observability and 08 Deployment Strategies. The DevOps map shows how they fit together.
Linux and Shell Questions
Reference: page 01 Linux & Shell Essentials.
- Q: What does load average mean, and is a load of 8 bad?
It is the average number of tasks that are runnable or in uninterruptible sleep (usually waiting on disk I/O) over 1, 5 and 15 minutes. Whether 8 is bad depends on the number of CPU cores: 8 on an 8-core machine is roughly fully busy, 8 on 2 cores means a queue. A high load with low CPU use points to processes blocked on I/O. - Q: A disk shows free space in
df -hbut writes fail with "No space left on device". Why?
The filesystem may be out of inodes (checkdf -i), typically from millions of tiny files. Another cause is a deleted file still held open by a running process (find withlsof +L1), whose space is released only when the process closes it or exits. - Q: What is a zombie process and how do you get rid of it?
A zombie has finished running but its parent has not yet read its exit status withwait(), so its entry stays in the process table. It uses no CPU or memory beyond the entry and cannot be killed (it is already dead). Fix the parent so it reaps children, or kill or restart the parent soinitadopts and reaps them. - Q: Difference between
kill,kill -9, and why prefer the first?killsends SIGTERM, which a process can catch to shut down gracefully (finish requests, flush, close files).kill -9sends SIGKILL, which cannot be caught or ignored: the kernel ends the process immediately, with no cleanup. Use SIGTERM first and SIGKILL only if it does not exit. - Q: Explain file permissions
rwxr-x---and the numeric form.
Owner can read, write, execute; group can read and execute; others have nothing. Numeric: r=4, w=2, x=1, so owner 7, group 5, others 0, which is750. For directories,xmeans permission to enter and access entries. - Q: Hard link vs symbolic link?
A hard link is another directory entry for the same inode (same file, same data, cannot cross filesystems, survives deletion of the original name). A symbolic link is a small file containing a path; it can point anywhere, including to a directory or across filesystems, and breaks if the target is removed. - Q: How do you find which process is listening on port 8080?
ss -ltnp | grep 8080(orlsof -i :8080), which shows the PID and program.sudomay be needed to see processes of other users. - Q: How do you investigate a slow server?
Apply the USE method per resource: CPU (top,uptime,pidstat), memory (free -m, swapping invmstat), disk (iostat -x,df), network (ss,sar -n DEV). Then identify the process, and read its logs. Start with the question "what changed?"
Docker and Container Questions
Reference: page 02 Docker.
- Q: How is a container different from a virtual machine?
A VM virtualises hardware and runs a full guest operating system on a hypervisor. A container is an ordinary process (or group of processes) on the host kernel, isolated with namespaces (what it can see: PIDs, network, mounts) and limited with cgroups (how much CPU and memory it may use). Containers start faster and are lighter but share the host kernel, so isolation is weaker. - Q: What is the difference between an image and a container?
An image is a read-only, layered template (filesystem plus metadata). A container is a running (or stopped) instance of an image with a thin writable layer on top. Many containers can run from one image. - Q: How does Docker layer caching work, and how do you order a Dockerfile for it?
Each instruction creates a layer; a layer is reused only if the instruction and everything before it are unchanged. Put rarely changing steps first (base image, dependency download) and frequently changing ones last (copying source), so a code change does not reinstall dependencies. - Q: What is a multi-stage build and why use it?
SeveralFROMstages in one Dockerfile; the build stage has the compiler and tools, and the final stage copies only the built artefact onto a small runtime image. The result is smaller, has fewer vulnerabilities and does not ship build tools. - Q:
CMDvsENTRYPOINT?ENTRYPOINTdefines the executable that always runs;CMDprovides default arguments (or a default command if no entrypoint) that are easy to override atdocker run. A common pattern isENTRYPOINT ["java", "-jar", "app.jar"]withCMDfor default arguments. Prefer the exec (JSON array) form so the process receives signals directly. - Q: Why does a container sometimes ignore
docker stopand take 10 seconds to die?
The process is PID 1 in its namespace and may not handle SIGTERM (or was started through a shell form that does not forward it). After the grace period Docker sends SIGKILL. Use the exec form, handle SIGTERM, or add an init process such as--init. - Q: Volumes vs bind mounts?
A volume is managed by Docker, stored in its own area and independent of host paths: the right choice for persistent data. A bind mount maps a specific host path into the container: convenient for development, but it ties the container to the host layout and permissions. - Q: A container exits with code 137. What does it mean?
The process was killed by SIGKILL (128 + 9). The most common reason is the out-of-memory killer after exceeding the memory limit; confirm indocker inspect(OOMKilled) or the node logs. It can also be a manualkill -9or a timeout after SIGTERM. For the JVM, remember the heap is only part of its memory use. - Q: Name container security practices you apply.
Run as a non-root user; use minimal, pinned base images and scan them; do not bake secrets into images; drop capabilities and use a read-only root filesystem where possible; set resource limits; sign and verify images.
Kubernetes Questions
Reference: pages 03 Kubernetes Core and 04 Kubernetes Operations.
- Q: Explain Pod, ReplicaSet and Deployment.
A Pod is the smallest deployable unit: one or more containers sharing network and storage. A ReplicaSet keeps a given number of identical Pods running. A Deployment manages ReplicaSets to provide declarative rolling updates and rollbacks of a stateless workload. - Q: What happens when you run
kubectl apply -f deployment.yaml?
kubectl sends the object to the API server, which authenticates, authorises, validates and stores it in etcd. The Deployment controller creates a ReplicaSet, which creates Pods; the scheduler assigns each Pod to a node; the kubelet on that node pulls the image and starts the containers through the container runtime; the endpoints controller adds ready Pods to Service endpoints. - Q: Liveness vs readiness vs startup probes?
Liveness: is the container stuck? On failure the kubelet restarts it. Readiness: can it take traffic? On failure the Pod is removed from Service endpoints but not restarted. Startup: gives slow-starting applications time, and holds off the other probes until it succeeds. A common mistake is a liveness probe that checks dependencies, so a database outage restarts every Pod. - Q: Requests vs limits?
Requests are what the scheduler reserves for placement and the basis for QoS; limits are the ceiling enforced at runtime. Exceeding the CPU limit throttles the container; exceeding the memory limit gets it OOM-killed. Setting requests too low causes noisy neighbours and evictions; too high wastes capacity. - Q: Service types?
ClusterIP(default): a stable virtual IP reachable inside the cluster.NodePort: also opens a port on every node.LoadBalancer: asks the cloud for an external load balancer.ExternalName: a DNS alias. A headless Service (clusterIP: None) returns the Pod IPs directly, used with StatefulSets. - Q: Deployment vs StatefulSet?
A Deployment treats Pods as interchangeable. A StatefulSet gives each Pod a stable identity (ordinal name, stable DNS name) and its own persistent volume, and starts and stops them in order: for databases and other stateful systems. - Q: ConfigMap vs Secret. Are Secrets encrypted?
Both inject configuration. Secrets are intended for sensitive data and have separate access control, but by default they are only base64-encoded in etcd, not encrypted. Enable encryption at rest and tight RBAC, or use an external secrets manager. - Q: How does the Horizontal Pod Autoscaler work?
It periodically compares a metric (CPU utilisation relative to requests by default, or custom/external metrics) with a target and adjusts the replica count of a Deployment between a minimum and maximum. It needs resource requests set and the metrics pipeline installed. - Q: A Pod is stuck in
Pending. What do you check?kubectl describe podand read the scheduler events: insufficient CPU or memory on nodes, node selectors/affinity or taints that match no node, an unbound PersistentVolumeClaim, or a quota. Fix the cause (capacity, constraints, storage) rather than the Pod. - Q:
ImagePullBackOffvsCrashLoopBackOff?ImagePullBackOff: the image cannot be pulled (wrong name or tag, missing credentials, registry unreachable).CrashLoopBackOff: the container starts but keeps exiting, and Kubernetes restarts it with a growing delay. Different layer, different debugging: events for the first, logs and exit code for the second. - Q: What does a PodDisruptionBudget do?
It limits how many replicas of an application may be down at once during voluntary disruptions such as node drains and upgrades, so maintenance cannot take the whole service down. It does not protect against crashes or node failures.
CI/CD Questions
Reference: page 05 CI/CD Pipelines.
- Q: Continuous integration vs continuous delivery vs continuous deployment?
CI: every change is merged frequently and automatically built and tested. Continuous delivery: every passing change is automatically prepared as a releasable artefact, with a manual approval to release. Continuous deployment: every passing change goes to production automatically. - Q: What are the stages of a typical pipeline?
Checkout, build, unit tests, static analysis and dependency/security scans, package once into an immutable artefact (image), publish to a registry, deploy to a test environment, integration/end-to-end tests, then promote the same artefact to production with a gate or automatically. - Q: Why "build once, deploy many"?
Rebuilding per environment means production runs something that was never tested. Build one immutable artefact, version it by commit, and let configuration (not code) vary per environment, so what you tested is what you ship. - Q: How do you handle secrets in a pipeline?
Never in the repository or in plain pipeline variables echoed to logs. Use the CI system's secret store or an external manager (Vault, a cloud secrets service), inject at runtime with the least privilege, prefer short-lived credentials such as OIDC federation to the cloud over long-lived keys, and mask secrets in logs. - Q: Push-based deployment vs GitOps?
In push-based CD the pipeline runskubectl applyor similar against the cluster, so it needs cluster credentials. In GitOps the desired state lives in Git and an in-cluster agent (Argo CD, Flux) pulls and reconciles it, which gives an audit trail, drift correction and no cluster credentials in CI. - Q: What is a flaky test and what do you do about it?
A test that passes and fails without code changes (timing, shared state, external dependencies). It destroys trust in the pipeline. Quarantine it quickly, find the root cause (isolate state, remove sleeps, fake external services), and track flake rate; do not just rerun until green. - Q: Trunk-based development vs long-lived feature branches?
Trunk-based development merges small changes into main at least daily, using feature flags to hide unfinished work. It keeps integration continuous and merges small. Long-lived branches delay integration and make merges and releases risky.
Infrastructure as Code (Terraform) Questions
Reference: page 06 Infrastructure as Code.
- Q: What is Terraform state and why does it matter?
State is a file mapping the resources in your configuration to the real objects they created, plus metadata. Terraform compares configuration, state and reality to compute a plan. Lose or corrupt it and Terraform may try to recreate existing resources or forget they exist. - Q: Where should state live in a team, and how is it protected?
In a remote backend (for example an S3 bucket with versioning and encryption) shared by the team, with state locking so two applies cannot run at once. For the S3 backend, locking is enabled withuse_lockfile; the older DynamoDB table setting (dynamodb_table) is deprecated in the current documentation. State can contain secrets, so restrict access to it. - Q: Explain
terraform planandapply.plancomputes and shows the changes needed to reach the desired configuration without changing anything.applyexecutes them (optionally from a saved plan file, which guarantees you apply exactly what was reviewed). In CI, run plan on pull requests and apply after approval. - Q: What is drift and how do you handle it?
Drift is when real infrastructure differs from state/configuration, usually from manual console changes.terraform planreveals it. Either import/accept the change into code or let Terraform revert it, and reduce the cause with restricted console permissions. - Q:
countvsfor_each?countcreates N copies indexed by number: removing an item from the middle shifts the indices and can recreate resources.for_eachcreates one instance per key of a map or set, so each has a stable address and adding or removing one does not affect the others. Preferfor_eachfor collections. - Q: What are modules and when do you write one?
A module is a reusable group of resources with inputs and outputs. Write one when you repeat the same pattern (a VPC, a service with its IAM role and alarms), to enforce standards and cut copy and paste; version modules and avoid over-abstraction. - Q: How do you bring an existing resource under Terraform?
Write the resource configuration and import it into state (terraform import, or animportblock in recent versions), then runplanuntil it shows no changes. - Q: Declarative vs imperative infrastructure?
Declarative (Terraform, CloudFormation) describes the desired end state and the tool computes the steps; it is idempotent and reviewable. Imperative scripts describe steps, which are flexible but easy to get wrong when run twice or from an unexpected starting state.
Observability Questions
Reference: page 07 Observability.
- Q: Metrics, logs and traces: when do you use each?
Metrics to detect and alert (cheap, aggregated); traces to find which service or call is slow or failing; logs for the exact detail of one case. A trace id in every log line links them. - Q: What is the RED method and what is USE?
RED (rate, errors, duration) is for request-driven services. USE (utilisation, saturation, errors) is for resources such as CPU, disks and connection pools. Together with latency, traffic, errors and saturation (the four golden signals) they tell you what to dashboard first. - Q: Why not alert on average latency?
Averages hide the tail: a mean of 100 ms can coexist with a p99 of seconds. Alert on percentiles from a histogram, or on the fraction of requests slower than a threshold, ideally tied to an SLO. - Q: What is cardinality and which label is a classic mistake?
The number of distinct label combinations, each a separate time series. Auser_idorrequest_idlabel creates a series per value and exhausts Prometheus memory. Keep unbounded values in logs and traces. - Q: How do SLO burn-rate alerts work?
Burn rate is the observed error ratio divided by the ratio the SLO allows. For a 99.9 percent SLO a burn rate of 14.4 sustained for an hour consumes 2 percent of a 30-day budget. Page when both a long window (1 hour) and a short one (5 minutes) exceed it: significant and still happening. - Q: How do you expose Spring Boot metrics to Prometheus?
Add Actuator andmicrometer-registry-prometheus, exposeprometheusinmanagement.endpoints.web.exposure.include, and scrape/actuator/prometheus. - Q: How would you correlate one request across ten services?
Propagate the W3Ctraceparentheader (OpenTelemetry or Micrometer Tracing does this), put the trace id in every log line through the MDC, and look it up in the trace backend and the log backend.
Deployment Strategy Questions
Reference: page 08 Deployment Strategies.
- Q: Rolling, blue/green, canary: choose one for a payments service and defend it.
Canary with automated metric gates, behind a feature flag for new behaviour: the blast radius is small, rollback is a weight change, and payment success rate and latency are measurable. Blue/green is the alternative when instant rollback and pre-release testing matter more than capacity cost. - Q: How do you deploy a database column rename with zero downtime?
Expand and contract: add the new column, dual-write, backfill in batches, switch reads, and drop the old column in a later release. Never rename in place while two application versions run. - Q: When is rollback the wrong choice?
When data or schema has moved on irreversibly (a destructive migration, one-way conversion) so the old version cannot run against it; then roll forward with a minimal fix or disable the feature with a flag. - Q: What is the difference between a canary and an A/B test?
A canary asks whether the new version is healthy, on a percentage of requests, for minutes or hours. An A/B test asks which variant performs better on a business metric, with sticky user cohorts, for days or weeks. - Q: How do you separate deployment from release?
Ship the code behind a feature flag that is off, deploy it, verify, then enable it gradually per cohort and turn it off instantly if needed. Remove the flag when done.
Scenario Questions with Outlines of Strong Answers
Scenario questions have no single right answer. Interviewers listen for a method, prioritisation (restore service first) and verification. Each outline below lists what a strong answer covers, in a sensible order; adapt it to your own experience and say what you would check first.
"A new deployment went out and one pod keeps restarting. What do you do?"
kubectl get pods: is it one pod or all of them, and did it start with the deploy? If the whole service is down and the deploy is the cause, kubectl rollout undo first, debug afterkubectl describe pod <pod>: the last state, exit code and reason (Error, OOMKilled), restart count, and events. Then kubectl logs <pod> --previous for the crashed container's outputOOMKilled: memory limit too low or a leak. Exit 1 with a stack trace: application error (bad config, missing env var or secret, cannot reach the database). Exit 0 for a long-running service: the process ends by itself (wrong command). Probe failures show in events: a liveness probe that is too aggressive kills a slow starterkubectl rollout history, diff the specs)kubectl rollout status plus the error and restart metrics. Add a test or alert so it is caught earlier next time- Jumping to "delete the pod" or "increase the memory" without reading the exit code and the previous logs. Deleting the pod only restarts the same failure.
"Developers complain CI takes 40 minutes. How do you speed it up?"
- Measure first: get per-stage and per-step timings; find the critical path and the biggest items. Do not optimise by feel.
- Parallelise: split independent stages (unit tests, lint, security scan, build) into parallel jobs; shard the test suite across runners.
- Cache: dependency caches (Maven, npm), Docker layer cache, build-tool incremental builds; order Dockerfile layers well.
- Do less: run only what a change affects (path filters, affected-module builds in a monorepo); move slow end-to-end and security scans to a nightly or post-merge stage while keeping a fast pull-request gate.
- Test pyramid: replace slow end-to-end tests with unit and contract tests where possible; fix or quarantine flaky tests that cause reruns.
- Infrastructure: bigger or autoscaled runners, a faster registry or artefact mirror close to the runners, avoid cold starts.
- Fail fast: cheap checks first (lint, compile) so bad changes stop early.
- Verify and keep it fast: set a target (for example, pull-request feedback in 10 minutes), publish the pipeline duration as a metric, and alert on regressions.
"We need to split a name column into two and release without downtime."
- State the constraint: during a rolling or canary deploy, old and new code run together against one database, so the schema must be compatible with both.
- Expand: add nullable
first_nameandlast_namein a migration run before the deploy (Flyway or Liquibase as a pipeline step or Job, not on every pod start). - Deploy v2 that writes both old and new columns and still reads the old one; roll out with rolling or canary and readiness probes; rollback stays safe.
- Backfill existing rows in small batches as a background job; monitor replication lag and locks.
- Deploy v3 that reads the new columns only; verify with metrics and logs that nothing reads the old column.
- Contract: drop the old column in a later release, once no running version needs it; this is the only irreversible step.
- Also cover: API and message compatibility, a feature flag for the new behaviour, a rollback plan decided before starting, and post-deploy verification (error ratio, latency, business KPIs).
"p99 latency of checkout went from 300 ms to 3 s. Walk me through it."
"We have 20 services in separate repositories (or one monorepo). Design the pipeline."
- Standardise: a shared pipeline template (reusable workflow, shared library, or includes) so each service has a few lines of configuration and platform changes roll out once.
- Per-service pipeline: build, unit tests, static analysis and dependency scan, build one immutable image tagged by commit, push to a registry, generate an SBOM and sign the image.
- Only build what changed (path filters in a monorepo; independent repositories avoid the problem).
- Contract tests (consumer-driven) between services instead of one giant end-to-end suite; a small set of end-to-end smoke tests on the shared environment.
- Environments and promotion: deploy to dev automatically, promote the same artefact to staging and production through GitOps (Argo CD or Flux watching an environment repository) with approvals for production.
- Safe releases: canary or rolling with automated metric gates (Argo Rollouts or Flagger), feature flags, expand/contract migrations, and automatic rollback on SLO burn.
- Security and secrets: short-lived OIDC credentials, secrets from a manager, least-privilege runners, image scanning and policy checks (admission control).
- Observability of the pipeline: DORA metrics (deployment frequency, lead time, change failure rate, time to restore), build duration and flake rate.
- Trade-offs to mention: monorepo versus polyrepo, shared templates versus team autonomy, cost of ephemeral environments, and keeping the pull-request feedback loop short.
"Alerts say the root filesystem is at 100 percent. What do you do?"
- Stabilise: check what is failing (databases and logs stop first); free a little safe space (rotate or compress old logs, clear package caches) to restore writes.
- Find the cause:
df -handdf -i(space or inodes), thendu -xh --max-depth=1 /walking down to the largest directories;lsof +L1for deleted files still held open by a process. - Typical culprits: unrotated logs, core dumps, container images and layers, a runaway temp directory, a database growing as expected.
- Fix properly: configure log rotation and retention, move data to its own volume, set limits, clean unused images in a scheduled job.
- Prevent: an alert on disk trending to full (
predict_linear) well before 100 percent, not just on a fixed threshold.
"The pull request plan shows -/+ (destroy and recreate) for the production database. What now?"
- Do not apply. Read the plan: which attribute forces replacement (shown as "forces replacement"), for example a changed name, engine or identifier.
- Find the reason: a rename in code, a module upgrade that changed addresses (use
movedblocks orterraform state mvto tell Terraform it is the same resource), or drift. - Protect:
lifecycle { prevent_destroy = true }on critical resources, deletion protection on the database itself, backups and snapshots verified, and restricted apply permissions with required approval for production. - Process: plan output reviewed in the pull request, apply from the pipeline only, policy checks (Sentinel/OPA or similar) that flag destroys of stateful resources.
"Every deployment causes a few seconds of 502 errors, although pods are healthy. Why and how do you fix it?"
- Hypotheses: terminating pods still receive traffic (endpoint removal races SIGTERM), requests killed mid-flight, new pods marked ready before the application can serve, or a load balancer with a long health-check or deregistration delay.
- Check: the timing of errors against pod termination and readiness events; the readiness probe path and what it checks; whether the app handles SIGTERM.
- Fix: a readiness probe that reflects real readiness,
minReadySeconds, graceful shutdown (server.shutdown=graceful, a longerterminationGracePeriodSeconds), apreStopdelay so the endpoints and load balancer drain first, andmaxUnavailable: 0with a surge. - Verify: repeat a deploy under load with an error-ratio panel on screen and confirm zero errors; then put that check into the pipeline.
Cheat Sheet and Recap
| Theme | Commands / ideas to have ready | Common trap |
|---|---|---|
| Linux debugging | top, free -m, df -h/df -i, iostat -x, ss -ltnp, journalctl, lsof | Blaming CPU when the problem is I/O or inodes |
| Docker | Image vs container, layer cache, multi-stage, exec-form CMD, non-root | Exit 137 means SIGKILL, often OOM |
| Kubernetes debug | describe, logs --previous, get events, rollout undo, top | Deleting a crashing pod instead of reading why |
| Probes | Liveness restarts, readiness gates traffic, startup protects slow starts | Liveness probes that check dependencies |
| Resources | Requests schedule, limits enforce (CPU throttles, memory kills) | No requests or limits at all |
| CI/CD | Build once, promote the same artefact; GitOps pull model | Rebuilding per environment |
| Terraform | Remote state with locking, plan before apply, for_each, import | Console changes causing drift; secrets in state |
| Observability | RED/USE, histograms, cardinality, burn-rate alerts | Averages and high-cardinality labels |
| Deployments | Rolling, blue/green, canary, flags, expand/contract | Incompatible schema and code in one release |
| Incidents | Stop the bleeding, scope, hypothesis, one change at a time, verify | Debugging before restoring service |
- Explain mechanisms, not tool names. "Kubelet restarts a container when liveness fails" beats "I use probes".
- Always give a method for debugging scenarios: stabilise, scope, hypothesis, test, fix, verify.
- Name the trade-off in every design answer (cost versus rollback speed, autonomy versus consistency).
- Quote concrete signals: exit codes, events, metrics, the specific command you would run first.
- Be honest about what you have not used; describe how you would find out.
Checked on 2026-10-31: the Terraform S3 backend documentation (state locking with use_lockfile, dynamodb_table marked deprecated) and the Kubernetes pod debugging documentation (Pending and crash troubleshooting flow). The remaining answers are well-established behaviour of Linux, Docker, Kubernetes and Terraform; where a default or a number matters it is stated on the topic page with its version. Verify details against the documentation of the versions you run.
- DevOps map - the learning path through all eight topic pages.
- 08 Deployment Strategies and 07 Observability - the two pages behind most scenario questions.
- Architecture: System Design - for the design half of senior interviews.