Scarpy is a great framework for scraping purpose. It has a very good structure for large projects and Oops style remove duplicate code.
```def parse(self, response):
yield FormRequest(posturl,
formdata{'username':'yourUserName','Password':'!Z6Dj2hqaR'},
callback=self.after_login)
def after_login(self, response):
yield Request("pageurl,callback=self.directory_page, meta={'page':1})
This Post is just for education Purpose.
This code shows how to get login in a website using Scrapy:-
yield FormRequest(posturl,
formdata{'username':'yourUserName','Password':'!Z6Dj2hqaR'},
callback=self.after_login)
def after_login(self, response):
yield Request("pageurl,callback=self.directory_page, meta={'page':1})
```
After this Scrapy takes care itself of session and cookies.
