One - One Code All

Blog Content

airflow的参数

Python 并行计算   2014-03-25 21:49:05

官网参考: https://airflow.apache.org/code.html?highlight=ds_nodash#macros



Default Variables

The Airflow engine passes a few variables by default that are accessible in all templates

VariableDescription
{{ ds }}the execution date as YYYY-MM-DD
{{ ds_nodash }}the execution date as YYYYMMDD
{{ prev_ds }}the previous execution date as YYYY-MM-DD. if {{ ds }} is 2016-01-08 and schedule_interval is @weekly,{{ prev_ds }} will be 2016-01-01.
{{ next_ds }}the next execution date as YYYY-MM-DD. if {{ ds }} is 2016-01-01 and schedule_interval is @weekly,{{ prev_ds }} will be 2016-01-08.
{{ yesterday_ds }}yesterday’s date as YYYY-MM-DD
{{ yesterday_ds_nodash }}yesterday’s date as YYYYMMDD
{{ tomorrow_ds }}tomorrow’s date as YYYY-MM-DD
{{ tomorrow_ds_nodash }}tomorrow’s date as YYYYMMDD
{{ ts }}same as execution_date.isoformat()
{{ ts_nodash }}same as ts without - and :
{{ execution_date }}the execution_date, (datetime.datetime)
{{ prev_execution_date }}the previous execution date (if available) (datetime.datetime)
{{ next_execution_date }}the next execution date (datetime.datetime)
{{ dag }}the DAG object
{{ task }}the Task object
{{ macros }}a reference to the macros package, described below
{{ task_instance }}the task_instance object
{{ end_date }}same as {{ ds }}
{{ latest_date }}same as {{ ds }}
{{ ti }}same as {{ task_instance }}
{{ params }}a reference to the user-defined params dictionary which can be overridden by the dictionary passed through trigger_dag -c if you enableddag_run_conf_overrides_params` in ``airflow.cfg
{{ var.value.my_var }}global defined variables represented as a dictionary
{{ var.json.my_var.path }}global defined variables represented as a dictionary with deserialized JSON object, append the path to the key within the JSON object
{{ task_instance_key_str }}a unique, human-readable key to the task instance formatted {dag_id}_{task_id}_{ds}
{{ conf }}the full configuration object located atairflow.configuration.conf which represents the content of yourairflow.cfg
{{ run_id }}the run_id of the current DAG run
{{ dag_run }}a reference to the DagRun object
{{ test_mode }}whether the task instance was called using the CLI’s test subcommand

Note that you can access the object’s attributes and methods with simple dot notation. Here are some examples of what is possible:{{ task.owner }}, {{ task.task_id }}, {{ ti.hostname }}, … Refer to the models documentation for more information on the objects’ attributes and methods.

The var template variable allows you to access variables defined in Airflow’s UI. You can access them as either plain-text or JSON. If you use JSON, you are also able to walk nested structures, such as dictionaries like:{{ var.json.my_dict_var.key1 }}

Macros

Macros are a way to expose objects to your templates and live under themacros namespace in your templates.

A few commonly used libraries and methods are made available.

VariableDescription
macros.datetimeThe standard lib’s datetime.datetime
macros.timedeltaThe standard lib’s datetime.timedelta
macros.dateutilA reference to the dateutil package
macros.timeThe standard lib’s time
macros.uuidThe standard lib’s uuid
macros.randomThe standard lib’s random




上一篇:Airbnb开源项目airflow
下一篇:airflow删除某个dag

The minute you think of giving up, think of the reason why you held on so long.