Demo entry 6869967

dag test


Submitted by anonymous on Aug 21, 2019 at 09:19
Language: Python 3. Code size: 523 Bytes.

def test_import_time():
    """Test to ensure all dags are loaded less than threshold time

    dag_bag = DagBag()
    stats = dag_bag.dagbag_stats

    slow_dags = filter(lambda d: d.duration > LOAD_SECOND_THRESHOLD, stats)
    dags = ','.join(map(lambda d: f"{d.file[1:]} - {d.duration}", slow_dags))
    nl = '\n'

    if dags:
        # pylint: disable=C1801
        assert len(dags.split(',')) == 0, f"The following files take more than {LOAD_SECOND_THRESHOLD}s to load: {dags.replace(',', nl)}"

This snippet took 0.00 seconds to highlight.

Back to the Entry List or Home.

Delete this entry (admin only).