Currently available quadrotor simulators have a rigid and highly-specialized structure, either are they really fast, physically accurate, or photo-realistic. In this work, we propose a paradigm-shift in the development of simulators, moving the trade-off between accuracy and speed from the developers to the end-users. We release a new modular quadrotor simulator, Flightmare. Flightmare is composed of two main components, a configurable rendering engine built on Unity and a flexible physics engine for dynamics simulation. Those two components are totally decoupled and can run independently from each other. Flightmare comes with several desirable features, (i) a large multi-modal sensor suite, including an interface to extract the 3D point-cloud of the scene; (ii) an API for reinforcement learning which can simulate hundreds of quadrotors in parallel; and (iii) an integration with a virtual-reality headset for interaction with the simulated environment. Flightmare can be used for various applications, including path-planning, reinforcement learning, visual-inertial odometry, deep learning, human-robot interaction, etc.