Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Top Posters

Who's Online (0)

Powered by Vanilla. Made with Bootstrap.
Python Google Scan No Precoded Modules
  • Xin
    Posts: 3,251
    Hey guys im trying to make a google scanner with no precoded modules, i did it in visual basic a while back and it was really simple, a simple regex but i think google have changed their code a bit and it doesnt work, anyway heres what i think the python equivalent of my old code was and can you guys try help me get this working , thanks.

    import urllib2
    import re
    import threading
    url = urllib2.urlopen('http://www.google.co.uk/#hl=en&xhr=t&q=xinapse&cp=5&pf=p&sclient=psy&site=&source=hp&aq=0&aqi=&aql=&oq=xinaps&pbx=1&fp=68704d3128309f03')
    source = url.read()

    r1 = re.compile('/<h2 class=(.*?)><a href=\\"(.*?)\\" class=(.*?)>/')




    links = r1.findall(source)

    for link in links:
    print link



    Edit ive figure out my problem, the source from url.read is the google homepage source even though i passed a seearch string, any ideas?
    Xin
  • sangf
    Posts: 203
    mm.. not sure what google is doing but i can't even see the results in the source code at all, in fact i searched it for '<a href=' and it only displayed google specific links such as images, shopping, news, etc. however in the ssl version of google i can see it in the source, and it uses h3 with some other differences than the version you tried matching.

    oh well, ssl only example, seems to work, not fully tested:

    r1 = re&#46;compile(r'&lt;h3 class=\"r\"&gt;&lt;a href=&#91;\'\"&#93;?(&#91;^\'\" &gt;&#93;+)')
  • Heres a quick example I whipped up using simplejson:

    import urllib, simplejson, sys
    def searchgoogle(string)&#58;
    msg string
    searchquery = urllib&#46;urlencode({'q'&#58;msg})
    url = ('http&#58;//ajax&#46;googleapis&#46;com/ajax/services/search/web?v=1&#46;0&' + searchstring
    results = urllib&#46;urlopen(url)
    readrs = simplejson&#46;loads(results&#46;read())
    searchresults = readrs&#91;'responseData'&#93; &#91;'results'&#93;
    for x in searchresults&#58;
    titlex = x &#91;'title'&#93;&#46;replace('&lt;b&gt;','')
    title = titlex&#46;replace('&lt;/b&gt;','')
    msg = title + '&#58; ' + x&#91;'url'&#93;
    print msg

    searchgoogle(\"iexploit\")


    Output:

    iExploit - Home&#58; http&#58;//www&#46;iexploit&#46;org/
    iExploit&#58; http&#58;//www&#46;iexploit&#46;org/community/misc&#46;php%3Faction%3Dmarkread
    YouTube - iExploit&#39;s Channel&#58; http&#58;//www&#46;youtube&#46;com/user/iExploit
    iExploit (iExploitTweet) on Twitter&#58; http&#58;//twitter&#46;com/iexploittweet

    http://pypi.python.org/pypi/simplejson/

    Enjoy :)
  • Xin
    Posts: 3,251
    said:


    Heres a quick example I whipped up using simplejson:


    import urllib, simplejson, sys
    def searchgoogle(string)&#58;
    msg string
    searchquery = urllib&#46;urlencode({'q'&#58;msg})
    url = ('http&#58;//ajax&#46;googleapis&#46;com/ajax/services/search/web?v=1&#46;0&' + searchstring
    results = urllib&#46;urlopen(url)
    readrs = simplejson&#46;loads(results&#46;read())
    searchresults = readrs&#91;'responseData'&#93; &#91;'results'&#93;
    for x in searchresults&#58;
    titlex = x &#91;'title'&#93;&#46;replace('&lt;b&gt;','')
    title = titlex&#46;replace('&lt;/b&gt;','')
    msg = title + '&#58; ' + x&#91;'url'&#93;
    print msg

    searchgoogle(\"iexploit\")


    Output:

    iExploit - Home&#58; http&#58;//www&#46;iexploit&#46;org/
    iExploit&#58; http&#58;//www&#46;iexploit&#46;org/community/misc&#46;php%3Faction%3Dmarkread
    YouTube - iExploit&#39;s Channel&#58; http&#58;//www&#46;youtube&#46;com/user/iExploit
    iExploit (iExploitTweet) on Twitter&#58; http&#58;//twitter&#46;com/iexploittweet

    http://pypi.python.org/pypi/simplejson/

    Enjoy :)


    Loads of errors for me also i dont have the package simple json how do i install it?
    Xin
  • said:


    said:


    Heres a quick example I whipped up using simplejson:


    import urllib, simplejson, sys
    def searchgoogle(string)&#58;
    msg string
    searchquery = urllib&#46;urlencode({'q'&#58;msg})
    url = ('http&#58;//ajax&#46;googleapis&#46;com/ajax/services/search/web?v=1&#46;0&' + searchstring
    results = urllib&#46;urlopen(url)
    readrs = simplejson&#46;loads(results&#46;read())
    searchresults = readrs&#91;'responseData'&#93; &#91;'results'&#93;
    for x in searchresults&#58;
    titlex = x &#91;'title'&#93;&#46;replace('&lt;b&gt;','')
    title = titlex&#46;replace('&lt;/b&gt;','')
    msg = title + '&#58; ' + x&#91;'url'&#93;
    print msg

    searchgoogle(\"iexploit\")


    Output:

    iExploit - Home&#58; http&#58;//www&#46;iexploit&#46;org/
    iExploit&#58; http&#58;//www&#46;iexploit&#46;org/community/misc&#46;php%3Faction%3Dmarkread
    YouTube - iExploit's Channel&#58; http&#58;//www&#46;youtube&#46;com/user/iExploit
    iExploit (iExploitTweet) on Twitter&#58; http&#58;//twitter&#46;com/iexploittweet

    http://pypi.python.org/pypi/simplejson/

    Enjoy :)


    Loads of errors for me also i dont have the package simple json how do i install it?

    What OS are you running? If you're running *nix you can use a package installer. Heres how I did it:
    sudo aptitude install python-simplejson