Throw exception in DAG if status not 2xx

[anisbhsl]

This issue was encountered when we were trying to ingest fresh collections with veda_discover DAG. The DAG operation was a SUCCESS but no items were ingested by the ingestor API.

Further investigation through Cloudwatch logs revealed that the POST /ingestions endpoint was sending 422 response because it didn't have access to the S3 bucket it was supposed to have. This was ignored by airflow and it continued as a success.

Add an exception handler in airflow that fails the job if response status is other than the range of 2xx.

AC

• Add exception handler to throw error if status is 4xx or 5xx