Scrapy keeps hanging on depth limit

I'm trying to crawl a list of sites and have set a depth limit of 5. Scrapy is not able to crawl for more than a few minutes without getting an endless number of these messages: DEBUG: Ignoring link (depth > 5) which seams to go on for hours on the same site. Am I misunderstanding the way that the depth middleware should be used or is this a problem with scrapy? The code prevents the link extractor from navigating outside of the site by creating a list of rules using the site URLs. I'm including the code for it below because I suspect it may be part of the problem.

class EScraper(CrawlSpider):
    name = "emails"
    allowed_domains = []
    di = datainterface.DataInterface()
    dfn = di.getRowsBySIC('5511')
    rules = []
    for index, row in dfn.iterrows():
        allowed_domains.append(row['Website'].lower())
        lx = LinkExtractor()
        lx.allow_domains.add(row['Website'].lower())
        # lx.allow = re.compile(row['Website'].lower())
        rule = Rule(link_extractor=lx, follow=True, callback='landed',
                    cb_kwargs={'index': index, 'depth': 0, 'firstName': row['LastName'],
                                'lastName': row['FirstName'], 'found': {},
                               'title': row['Title'], 'main': row['Website']}) 
        rules.append(rule)