Quantcast
Channel: Active questions tagged feedparser - Stack Overflow
Viewing all articles
Browse latest Browse all 105

Python Feedparser script freezes - asynchronous tasks too

$
0
0

I wrote a Python script to fetch rss (using feedparser library) and catch regex matches.

I had problems in a punctual manner (sometimes script didn't fetch feed anymore).

I tried to add an asynchronous function to kill and restart the script if no log was written in about 25 minutes. But no more result.

  • Script seems to be stuck in a void
    • No more rss fetch
    • No more log monitoring
    • No error
async def parse_rss():    while True:        try:            feed = feedparser.parse(rss_url)            try:                while feed.status == 200:                    logger.info("checked: 200")                    catchregex & do things                    await asyncio.sleep(600)                    feed.status = 0            except Exception as e:                logger.error(f"11 - {e}")                await asyncio.sleep(30)                continue        except Exception as e:            logger.error(f"44 - {e}")            continueasync def monitor_log_file(log_file, timeout):    while True:        logger.info("log monitor")        last_modified = os.path.getmtime(log_file)        current_time = time.time()        if current_time - last_modified > timeout:            logger.error("No news - kill the script.")            subprocess.call(["python3", __file__])            break        await asyncio.sleep(timeout) # call these functions by:    await asyncio.gather(parse_rss(),monitor_log_file(log_file, timeout))
  • Logfile example
2023-05-23 23:11:31,085 - INFO - checked: 2002023-05-23 23:21:32,085 - INFO - checked: 2002023-05-23 23:31:33,073 - INFO - checked: 2002023-05-23 23:41:34,182 - INFO - checked: 2002023-05-23 23:41:47,825 - INFO - log monitor2023-05-23 23:51:35,082 - INFO - checked: 2002023-05-23 00:01:36,064 - INFO - checked: 2002023-05-23 00:11:38,013 - INFO - checked: 2002023-05-23 00:21:38,972 - INFO - checked: 200Frozen here, but no error

Any ideas of the problem?

Thanks


Viewing all articles
Browse latest Browse all 105

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>