![]() # or more contributor license agreements. 2) I then ran the command docker-compose -f docker-compose-LocalExecutor.yml up -d to start an Airflow webserver and the Postgres database. Please refer to the Resources section in the Docker for Windows or Docker for Mac documentation for more information. This will run the image and all the containers under it. Depending on your OS, you may need to configure Docker to use at least 4.00 GB of memory for the Airflow containers to run properly. It looks like there is no connectivity between containers, but I dont know how to fix it. ![]() Once the image is built successfully, we can run this image with a docker-compose up command. If you want to use Ad hoc query, make sure youve configured connections: Go to Admin -> Connections and Edit airflowdb set this values (equivalent to values. Im trying to set up airflow instance using docker-compose as described in official docs and Im stuck at airflow-init part. docker build -t airflow-sqlserver -f Dockerfile. Apache airflow is a great tool for orchestrating workflows and data. It can take a while depending on the number of libraries we are installing. With DAG('docker_operator_dag', default_args=default_args, schedule_interval="5 * * * *", catchup=False) as dag:Īlso, I'm using Windows 10, I've tried adding the following in volumes: - "/var/run/docker.sock:/var/run/docker.sock" and I've succeeded in Ubuntu, but in Windows doesn't.Īs requested, follow the docker-compose file: # Licensed to the Apache Software Foundation (ASF) under one Connect Snowflake with Airflow on Docker to Build A Data Extraction Pipeline for Free. 'description' : 'Use of the DockerOperator', I'm using the Airflow docker-compose found here and trying to run the following DAG: from datetime import datetime, timedeltaįrom _operator import BashOperatorįrom _operator import DockerOperatorįrom _operator import BranchPythonOperatorįrom _operator import DummyOperator ![]() Follows the error launched: *** Reading local file: /opt/airflow/logs/docker_operator_dag/docker_command_hello/T02:40:13.171571+00:00/2.log I'm trying to run a DAG that calls a docker container and executes a command inside it, but Airflow cannot execute the task. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |