[Solved] Apache Airflow: Control over logging [Disable/Adjust logging level]

I am using Airflow installed using pip

I would like to limit the logging to ERROR level for the workflow being executed by the scheduler. Could not find anything beyond setting log files location in the settings.

py file.

Also the online resources led me to this google group discussion here but not much info here as well

Any idea how to control logging in Airflow?

Solution #1:

The logging functionality and its configuration will be changed in version 1.9 with this commit

Respondent: Dimo Boyadzhiev
Solution #2:

I tried below work around and it seems to be working to set LOGGING_LEVEL outside of

  • Update
    • Remove or comment line:
      LOGGING_LEVEL = logging.INFO
    • Add line:
      LOGGING_LEVEL = os.path.expanduser(conf.get('core', 'LOGGING_LEVEL'))
  • Update airflow.cfg configuration file:
    • Add line under [core]:
      logging_level = WARN
    • Restart webserver and scheduler services
  • Use environment vaiable AIRFLOW__CORE__LOGGING_LEVEL=WARN.

See the official docs for details.

Respondent: Kiran
Solution #3:

As @Dimo Boyadzhiev pointed the change, Adding the more info path for documentaion.

File - $AIRFLOW_HOME/airflow.cfg

# Logging level
logging_level = INFO
fab_logging_level = WARN
Respondent: Chandan Nayak
Solution #4:

Only solution I am aware of is changing LOGGING_LEVEL in file. Default level is set to INFO.

AIRFLOW_HOME = os.path.expanduser(conf.get('core', 'AIRFLOW_HOME'))
SQL_ALCHEMY_CONN = conf.get('core', 'SQL_ALCHEMY_CONN') 
DAGS_FOLDER = os.path.expanduser(conf.get('core', 'DAGS_FOLDER'))
Respondent: Kiran
The answers/resolutions are collected from stackoverflow, are licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0 .

Most Popular

To Top
India and Pakistan’s steroid-soaked rhetoric over Kashmir will come back to haunt them both clenbuterol australia bossier man pleads guilty for leadership role in anabolic steriod distribution conspiracy