Python Development Mode
***********************

New in version 3.7.

The Python Development Mode introduces additional runtime checks that
are too expensive to be enabled by default. It should not be more
verbose than the default if the code is correct; new warnings are only
emitted when an issue is detected.

It can be enabled using the "-X dev" command line option or by setting
the "PYTHONDEVMODE" environment variable to "1".

See also Python debug build.


Effects of the Python Development Mode
**************************************

Enabling the Python Development Mode is similar to the following
command, but with additional effects described below:

   PYTHONMALLOC=debug PYTHONASYNCIODEBUG=1 python3 -W default -X faulthandler

Effects of the Python Development Mode:

* Add "default" warning filter. The following warnings are shown:

  * "DeprecationWarning"

  * "ImportWarning"

  * "PendingDeprecationWarning"

  * "ResourceWarning"

  Normally, the above warnings are filtered by the default warning
  filters.

  It behaves as if the "-W default" command line option is used.

  Use the "-W error" command line option or set the "PYTHONWARNINGS"
  environment variable to "error" to treat warnings as errors.

* Install debug hooks on memory allocators to check for:

  * Buffer underflow

  * Buffer overflow

  * Memory allocator API violation

  * Unsafe usage of the GIL

  See the "PyMem_SetupDebugHooks()" C function.

  It behaves as if the "PYTHONMALLOC" environment variable is set to
  "debug".

  To enable the Python Development Mode without installing debug hooks
  on memory allocators, set the "PYTHONMALLOC" environment variable to
  "default".

* Call "faulthandler.enable()" at Python startup to install handlers
  for the "SIGSEGV", "SIGFPE", "SIGABRT", "SIGBUS" and "SIGILL"
  signals to dump the Python traceback on a crash.

  It behaves as if the "-X faulthandler" command line option is used
  or if the "PYTHONFAULTHANDLER" environment variable is set to "1".

* Enable asyncio debug mode. For example, "asyncio" checks for
  coroutines that were not awaited and logs them.

  It behaves as if the "PYTHONASYNCIODEBUG" environment variable is
  set to "1".

* Check the *encoding* and *errors* arguments for string encoding and
  decoding operations. Examples: "open()", "str.encode()" and
  "bytes.decode()".

  By default, for best performance, the *errors* argument is only
  checked at the first encoding/decoding error and the *encoding*
  argument is sometimes ignored for empty strings.

* The "io.IOBase" destructor logs "close()" exceptions.

* Set the "dev_mode" attribute of "sys.flags" to "True".

The Python Development Mode does not enable the "tracemalloc" module
by default, because the overhead cost (to performance and memory)
would be too large. Enabling the "tracemalloc" module provides
additional information on the origin of some errors. For example,
"ResourceWarning" logs the traceback where the resource was allocated,
and a buffer overflow error logs the traceback where the memory block
was allocated.

The Python Development Mode does not prevent the "-O" command line
option from removing "assert" statements nor from setting "__debug__"
to "False".

The Python Development Mode can only be enabled at the Python startup.
Its value can be read from "sys.flags.dev_mode".

Changed in version 3.8: The "io.IOBase" destructor now logs "close()"
exceptions.

Changed in version 3.9: The *encoding* and *errors* arguments are now
checked for string encoding and decoding operations.


ResourceWarning Example
***********************

Example of a script counting the number of lines of the text file
specified in the command line:

   import sys

   def main():
       fp = open(sys.argv[1])
       nlines = len(fp.readlines())
       print(nlines)
       # The file is closed implicitly

   if __name__ == "__main__":
       main()

The script does not close the file explicitly. By default, Python does
not emit any warning. Example using README.txt, which has 269 lines:

   $ python3 script.py README.txt
   269

Enabling the Python Development Mode displays a "ResourceWarning"
warning:

   $ python3 -X dev script.py README.txt
   269
   script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>
     main()
   ResourceWarning: Enable tracemalloc to get the object allocation traceback

In addition, enabling "tracemalloc" shows the line where the file was
opened:

   $ python3 -X dev -X tracemalloc=5 script.py README.rst
   269
   script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>
     main()
   Object allocated at (most recent call last):
     File "script.py", lineno 10
       main()
     File "script.py", lineno 4
       fp = open(sys.argv[1])

The fix is to close explicitly the file. Example using a context
manager:

   def main():
       # Close the file explicitly when exiting the with block
       with open(sys.argv[1]) as fp:
           nlines = len(fp.readlines())
       print(nlines)

Not closing a resource explicitly can leave a resource open for way
longer than expected; it can cause severe issues upon exiting Python.
It is bad in CPython, but it is even worse in PyPy. Closing resources
explicitly makes an application more deterministic and more reliable.


Bad file descriptor error example
*********************************

Script displaying the first line of itself:

   import os

   def main():
       fp = open(__file__)
       firstline = fp.readline()
       print(firstline.rstrip())
       os.close(fp.fileno())
       # The file is closed implicitly

   main()

By default, Python does not emit any warning:

   $ python3 script.py
   import os

The Python Development Mode shows a "ResourceWarning" and logs a "Bad
file descriptor" error when finalizing the file object:

   $ python3 script.py
   import os
   script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>
     main()
   ResourceWarning: Enable tracemalloc to get the object allocation traceback
   Exception ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>
   Traceback (most recent call last):
     File "script.py", line 10, in <module>
       main()
   OSError: [Errno 9] Bad file descriptor

"os.close(fp.fileno())" closes the file descriptor. When the file
object finalizer tries to close the file descriptor again, it fails
with the "Bad file descriptor" error. A file descriptor must be closed
only once. In the worst case scenario, closing it twice can lead to a
crash (see bpo-18748 for an example).

The fix is to remove the "os.close(fp.fileno())" line, or open the
file with "closefd=False".