Data Pipeline with Buffering

Hard Python Concurrency
Solve this problem

Design a Python data pipeline with threaded buffering. The producer reads data from a sensor (simulated) and pushes into a bounded queue. The consumer thread processes and writes results to a file. Backpressure must be enforced.

Constraints

  • Use queue.Queue with maxsize.
  • producer() blocks when full.
  • consumer() blocks when empty.
  • Use sentinel None to stop.
  • Flush file operations cleanly.
  • Handle KeyboardInterrupt.
Starter codePython
1import threading, queue2# TODO