Scrapy Crawl Expired Domains

I'm using Scrapy to crawl differents websites, but actually my script follow every websites and add to database the domains and after I check with a PHP script the expired domain.

I would like someone able to help me to improve my script because the actual script is not optimised for my what I need !

I don't know why but the crawler jump immediately on the differents websites find on the "start url", It will be better if the script finish the scan the first website before jump on an other website.

And how can I check directly if the domain is expired before add it to the database ?

My Crawler :

from scrapy.spiders import CrawlSpider, Rule
from dirbot.settings import *
from scrapy.linkextractors.lxmlhtml import LxmlLinkExtractor
from scrapy.item import Item, Field
from urlparse import urlparse

class MyItem(Item):
    url= Field()

class someSpider(CrawlSpider):
    name = 'expired'
    start_urls = ['http://domain.com']

    rules = (Rule(LxmlLinkExtractor(allow=()), callback='parse_obj', follow=True),)

    def parse_obj(self,response):
        item = MyItem()
        item['url'] = []
        for link in LxmlLinkExtractor(allow='/.com|.fr|.net|.org|.info/i',deny = '/.jp|facebook|amazon|wordpress|blogspot|free.|google|yahoo|bing|znet|stackexchange|twitter|wikipedia/i').extract_links(response):
            parsed_uri = urlparse(link.url)
            url = '{uri.scheme}://{uri.netloc}/'.format(uri=parsed_uri)
        insert_table(url)