Advertisements

Blog Archives

#MacOsX : Terminal Cheat Sheet

If you are a *nix geek like me you can’t but love the command prompt.
One of the best tool to improve the plain old terminal is an utility called tmux. You can install through Homebrew.
Now, there are many commands to remember to play nicely with the terminal, and sometimes a little remind might be useful, that’s why cheat sheets exist.
Here is mine, enjoy.

Advertisements

#cURL : HOWTO [UPDATED]

You can use the cURL library and the curl command to design your own Request and explore the Response. There are many possible uses like e.g., API debug, web hacking, pen testing.
curl is a tool to transfer data from or to a server, using one of the supported protocols (e.g., FTP, GOPHER, HTTP, HTTPS, IMAP, LDAP, POP3, RTMP, SCP, SFTP, SMTP, TELNET). The command is designed to work without user interaction.
curl offers a busload of useful tricks like proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more. As you will see below, the number of features will make your head spin!
So curl is a truly powerful command, however it does at the cost of complexity. Here I will show some real-world use cases.

URL

The URL syntax is protocol-dependent. If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want. It will then default to HTTP but try other protocols based on often-used host name prefixes. For example, for host names starting with “ftp.” curl will assume you want to speak FTP.
You can specify multiple URLs or parts of URLs by writing part sets within braces as in:

curl en.wikipedia.org/wiki/{FTP,SCP,TELNET}

or you can get sequences of alphanumeric series by using [ ] as in:

curl forums.macrumors.com/showthread.php?t=[1673700-1673713]
curl numericals.com/file[1-100].txt
curl numericals.com/file[001-100].txt
curl letters.com/file[a-z].txt

Nested sequences are not supported, but you can use several ones next to each other:

curl any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html

You can specify any amount of URLs on the command line. They will be fetched in a sequential manner in the specified order.
You can specify a step counter for the ranges to get every Nth number or letter:

curl numericals.com/file[1-100:10].txt
curl letters.com/file[a-z:2].txt

Trace Dump

In order to analyze in depth what we send and receive we might save everything on a file, this is as easy as:

curl --trace-ascii DebugDump.txt URL

Save To Disk

If you want save the Response to disk you can use option -o <file>. If you are using {} or [] to fetch multiple documents, you can use ‘#‘ followed by a number in the specifier. That variable will be replaced with the current string for the URL being fetched. Remember to protect the URL from shell by adding quotes if you receive the error message internal error: invalid pattern type (0). Examples:

curl 'en.wikipedia.org/{FTP,TFTP,SFTP}' -o "#1.html"
curl arxiv.org/pdf/13[01-11].36[00-75].pdf -o "arXiv13#1.36#2.pdf"

Option -O writes output to a local file named like the remote file we get (only the file part of the remote file is used, the path is cut off). The remote file name to use for saving is extracted from the given URL, nothing else. Consequentially, the file will be saved in the current working directory. If you want the file saved in a different directory, make sure you change current working directory before you invoke curl:

curl -O arxiv.org/pdf/1301.3600.pdf

Only the file part of the remote file is used, the path is cut off, thus the file will be saved as 1301.3600.pdf.

Set HTTP Request Method

The curl default HTTP method, GET, can be set to any method you would like using the -X <command> option. The usual suspects POST, PUT, DELETE, and even custom methods, can be specified:

curl -X POST echo.httpkit.com

Normally you don’t need this option. All sorts of GET, HEAD, POST and PUT requests are rather invoked by using dedicated command line options.

Forms

Forms are the general way a web site can present a HTML page with fields for
the user to enter data in, and then press some kind of ‘submit’
button to get that data sent to the server. The server then typically uses
the posted data to decide how to act. Like using the entered words to search
in a database, or to add the info in a bug track system, display the entered
address on a map or using the info as a login-prompt verifying that the user
is allowed to see what it is about to see.
Using the -d option we can specify URL encoded field names and values:

curl -d "prefisso=051" -d "numero=806060" -d "Prosegui=Verifica" -d "form_name=verifica_copertura_ehiveco" http://www.ovus.it/verifica_copertura_ehiveco.php

A very common way for HTML based application to pass state information between pages is to add hidden fields to the forms. Hidden fields are already filled in, they aren’t displayed to the user and they get passed along just as all the other fields. To curl there is no difference at all, you just need to add it on the command line.

Set Request Headers

Request headers allow clients to provide servers with meta information about things such as authorization, capabilities, and body content-type. OAuth2 uses an Authorization header to pass access tokens, for example. Custom headers are set in curl using the -H option:

curl -H "Authorization: OAuth 2c4419d1aabeec" http://echo.httpkit.com
curl -H "Accept: application/json" -H "Authorization: OAuth 2c3455d1aeffc" http://echo.httpkit.com

Note that if you should add a custom header that has the same name as one of the internal ones curl would use, your externally set header will be used instead of the internal one. You should not replace internally set headers without knowing perfectly well what you’re doing. Remove an internal header by giving a replacement without content on the right side of the colon, as in: -H "Host:".
If you send the custom header with no-value then its header must be terminated with a semicolon, such as -H "X-Custom-Header;" to send "X-Custom-Header:".
curl will make sure that each header you add/replace is sent with the proper end-of-line marker, you should thus not add that as a part of the header content: do not add newlines or carriage returns, they will only mess things up for you.

Referer

A HTTP request may include a referer field (yes it is misspelled), which can be used to tell from which URL the client got to this particular resource. Some programs/scripts check the referer field of requests to verify that this wasn’t arriving from an external site or an unknown page. While this is a stupid way to check something so easily forged, many scripts still do it.
This can also be set with the -H, --header flag of course. When used with -L, --location you can append ";auto" to the --referer URL to make curl automatically set the previous URL when it follows a Location: header. The ";auto" string can be used alone, even if you don’t set an initial --referer.

curl -e google.com http://echo.httpkit.com

User Agent

To specify the User-Agent string to send to the HTTP server you can use --user-agent flag. To encode blanks in the string, surround the string with single quote marks. This can also be set with the -H, --header option of course. Many applications use this information to decide how to display pages. At times, you will see that getting a page with curl will not return the same page that you see when getting the page with your browser. Then you know it is time to set the User Agent field to fool the server into thinking you’re one of those browsers:

curl -A "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_2 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8H7 Safari/6533.18.5" http://echo.httpkit.com

Cookies

The way the web browsers do “client side state control” is by using cookies. Cookies are just names with associated contents. The cookies are sent to the client by the server. The server tells the client for what path and host name it wants the cookie sent back, and it also sends an expiration date and a few more properties.
When a client communicates with a server with a name and path as previously specified in a received cookie, the client sends back the cookies and their contents to the server, unless of course they are expired.
Many applications and servers use this method to connect a series of requests into a single logical session. To be able to use curl in such occasions, we must be able to record and send back cookies the way the web application expects them. The same way browsers deal with them.

It is supposedly the data previously received from the server in a "Set-Cookie:" line. The data should be in the format "NAME1=VALUE1; NAME2=VALUE2".
If no = symbol is used in the line, it is treated as a filename to use to read previously stored cookie lines from, which should be used in this session if they match. Using this method also activates the “cookie parser” which will make curl record incoming cookies too, which may be handy if you’re using this in combination with the -L, --location option. The file format of the file to read cookies from should be plain HTTP headers or the Netscape/Mozilla cookie file format. NOTE that the file specified with -b, --cookie is only used as input. No cookies will be stored in the file. To store cookies, use the -c, --cookie-jar option or you could even save the HTTP headers to a file using -D, --dump-header:

curl --cookie "name=whitehatty" http://echo.httpkit.com
curl -c cookies.txt http://www.facebook.com
sed -i '' s/#HttpOnly_\.facebook\.com/echo\.httpkit\.com/g cookies.txt
curl --cookie cookies.txt http://echo.httpkit.com
curl -b cookies.txt --cookie-jar newcookies.txt http://echo.httpkit.com
curl --dump-header headers_and_cookies http://www.facebook.com

Work In Progress…

Ok there are many more options, but I will stop here for now. I will add something in the future, so if you have any request (like using more real urls) just leave a comment.

#MacOsX : SSH SOCKS Proxy (SSH Tunnel Web Traffic)

After PRISM scandal you may feel the need to secure your connection and protect your privacy. Then it is a good idea to tunnel web traffic through a secure encrypted connection. This allows your traffic to traverse a local network without being visible to snoopers, even when visiting unencrypted web sites.

What you need:

  • a modern browser like Firefox, Chrome or Safari (they support SOCKS4 protocol)
  • ssh client (already installed in Mac Os X)
  • a shell account (with ssh access)

If you don’t have a shell account, you can find a free one HERE, or HERE, or HERE, or HERE.

To start the local proxy type:

ssh -D PORT user@host

where PORT is a local port between 1024 and 65535 (they do not require super user privileges), user is the username at the remote machine, and host is the identifier of the remote host.

That’s not enough, you need to configure the system to use the proxy. Go to System Preferences > Network > [select active interface] > Advanced... > Proxies and check SOCKS Proxy.
Then modify SOCKS Proxy Server info to use the PORT you chose before.

That’s it! From now on all connections on the active network interface will be tunneled through the proxy. 😎

NOTE1: you may want use proxy browser settings instead of system wide proxy settings, so you can tunnel only part of the traffic (e.g. the most sensitive one). This is easy, but the procedure slightly change between different browsers.

NOTE2: you may need to add the option -p HOST_PORT if the remote host doesn’t use the standard ssh port 22, e.g.:

ssh -D PORT user@host -p 666

#MacOsX : Mavericks Improve Virtual Machine Graphic Performance

I have made a short benchmark comparison of Parallels vs Fusion 5.0 HERE. VMWare Fusion has reached version 5.0.3, but it looks like VMWare has become lazy and those updates are not worth mentioning. However Apple released Mavericks recently, with an updated graphic stack which has slightly better graphics performance:

Component Lion/Mountain Lion Mavericks
Processor: 4.4 4.4
Memory (RAM): 5.5 5.5
Graphics: 5.0 5.0
Gaming graphics: 4.2 4.3
Primary hard disk: 7.7 7.7

Moreover, VMs seem to boot much faster under Mac Os X Mavericks.

N.B. you need to update VMWare Fusion to version 5.0.3 in order to have the best experience in Mavericks (or install VMWare Fusion 6).

#MacOsX : Show Hidden Files and Folders

In *NIX systems file and folders beginning with a dot (e.g., .name) are not visible in the Finder (also known as file browser). Since Mac Os X it’s a certified UNIX that’s also the case. If you use the terminal you can use the command:

ls -a

However, most people will use regular Finder. To enable view of hidden files in the Finder use this command:

defaults write com.apple.finder AppleShowAllFiles -bool TRUE

and then restart the finder with the following command:

killall Finder

To revert the changes use the same command, but replace TRUE with FALSE.

#MacOsX : vimrc

If you are looking to configure Vim you find the default configuration file in:

/usr/share/vim/vimrc

Copy and rename it in your home directory:

cp /usr/share/vim/vimrc ~/.vimrc

However it is bare minimal so it is better if you personalize it a bit. One very simple example is the following:

" Configuration file for vim
set modelines=0 " CVE-2007-2438

" Normally we use vim-extensions. If you want true vi-compatibility
" remove change the following statements
" Use Vim defaults instead of 100% vi compatibility
set nocompatible

" more powerful backspacing
set backspace=2

" Display line numbers on the left
set number

" Allow intelligent auto-indenting for each filetype
" and for "plugins that are filetype specific.
filetype indent plugin on

" Fallback when no filetype-specific indenting is enabled
set autoindent

" Enable syntax highlighting
syntax on

" Display the cursor position
set ruler

" Don't write backup file if vim is being called by "crontab -e"
au BufWrite /private/tmp/crontab.* set nowritebackup
" Don't write backup file if vim is being called by "chpass"
au BufWrite /private/etc/pw.* set nowritebackup

#UNIX : Send Messages to Logged Users

If you are used to play with multi users servers and you feel bored you can start a random chat with a logged user sending him a message (ok, actually it can be much more useful 😛 ) with the following command:

write username tty

where username and tty can be found using who command.
Then you can write the message and end it pressing Ctrl+D.

#MacOsX : Fix Mountain Lion Slow Shutdown

Ok, actually there are many good reasons to wait a bit of time before send a SIGKILL to processes (like giving them time to write things on disk or finish an upload on the iCloud), however the “slow” shutdown could be annoying, so try this:

sudo launchctl unload /System/Library/LaunchDaemons/com.apple.coreservices.appleevents.plist

sudo defaults write /System/Library/LaunchDaemons/com.apple.coreservices.appleevents ExitTimeOut -int 2

sudo launchctl load /System/Library/LaunchDaemons/com.apple.coreservices.appleevents.plist

This set the shutdown timeout to 2 sec instead of the default value (20).

#MacOsX : Web Hacking with Burp Suite

This is a legendary tool developed by the author of The Web Application Hacker’s Handbook: Finding and Exploiting Security Flaws (2nd edition).

Unfortunately there is no native Mac Os X version but Corsaire packed one. Since they updated the site, the link to download it provided in the aforementioned book is broken, so I will provide a new working one of the recently released version 1.5 1.6 of Burp Suite Free.
All rights reserved to Corsair and Portswigger.

The version at this link
DOWNLOAD HERE
is now outdated (it also requires JDK 6, which is no longer supported).
A free binary version for Mac OS has been made available:
Burp Suite Free Edition

See also here for a [much less powerful] alternative.

#MacOsX : Enable Quicklook Text Selection

Quicklook is a beautiful and powerful feature of Mac Os X (take a look here to know what is capable of) but it is somewhat weird that it doesn’t allow text selection.

To enable text selection in Quicklook:

defaults write com.apple.finder QLEnableTextSelection -boolean YES

killall Finder

 

#MacOsX : Show Remote Disks

There are Macs without disk drives. You can access CD/DVDs from another Mac over the network allowing remote disk.

This option allows you to always see remote drives within Finder:

defaults write com.apple.finder EnableODiskBrowsing -boolean YES

killall Finder

#MacOsX : How Cut & Paste Works in Mac OS X

Yep, Mac world is only about Drag & Drop but if you feel nostalgic of PC’s world you can move files the old way, just do this:

  • selct file/files and hit

    Command + C

  • move to another location an hit

    Command + Option + V

#MacOsX : Web Hacking with Paros

Paros is a web proxy that allows to intercept and modify all HTTP and HTTPS data between server and client, including cookies and form fields.

I will not explain how and why you use it, but if you are really interested, please take a look on the book: The Web Application Hacker’s Handbook: Discovering and Exploiting Security Flaws.

Unfortunately there is no native Mac Os X version but Corsaire packed one. Since they updated the site, the link to download it provided in the aforementioned book is broken, so I will provide a new working one. All rights reserved to Corsair and Paros Team.

DOWNLOAD HERE

Paros is no longer developed (it also requires JDK 6 which is not longer supported).
An alternative is a fork of it, Zed Attack Proxy, maintained by OWASP:
Zed Attack Proxy (ZAP)

 

#MacOsX : Reset NVRAM, PRAM and SMC

Sometimes after a software (system) or hardware upgrade even Macs can have some problems; if you have tried every solution without success you could try this.

Every Mac stores certain settings in a special memory area even if it is turned off. On Intel-based Macs, this is stored in memory known as NVRAM; on PowerPC-based Macs, this is stored in memory known as PRAM.

Resetting NVRAM and PRAM may solve lot of problems and could be required if you upgrade SSD or Mac firmware (more tips on SSDs here).

Another step to try is Resetting the System Management Controller (SMC). An SMC reset should only be attempted after all other standard troubleshooting has been performed.

 

 

#MacOsX : Disable Auto-Save and Versions in Mac OS X

Auto-Save and Versions are excellent features in Mac OS X, but some advanced users are annoyed by  them as they often don’t want to save intermediate versions of their work.
Moreover some apps write lots of data on disk (e.g. iMovie and iBooks Author) and this can shorten the life of SSD (look here for more tuning for SSDs).
If you know the name of the app plist you want to disable auto-save and Versions for, you can just plug the name into the defaults write command:

defaults write app-plist ApplePersistence -bool no

If you don’t know it then you can find it with the following command:

osascript -e 'id of application "NAME OF APP"'

Now if you enter the Versions window, auto-save list will be empty and there are no versions to restore to. You’ll probably want turn off File Locking too.

NOTE: some sandboxed apps require another command in addition:

defaults write app-plist AutosavingDelay -int 0

This is expecially true for TextEdit as it is the only Apple app that uses “old-style” autosaving and this causes issues with the sandbox in Lion/Mountain Lion.

NOTE2: It seems that the preference can be set globally but it may cause the login process to become very slow and possibly cause other unexpected behaviour:

defaults write -g ApplePersistence -bool no

#MacOsX : Turn Off File Locking

Mac OS X Lion has introduced automatic file locking for files that hasn’t been edited recently.

To Disable File Locking:

  • open System Preferences > Time Machine
  • click on Options
  • uncheck the box next to “Lock documents [2 weeks] after last edit” or modify the value as preferred
  • done 🙂

#VMware Fusion : Fix Ubuntu Linux “Host SMBus controller not enabled!” [UPDATED]

Ubuntu guest instances in VMware sometimes come up with the boot error message:

piix4_smbus 0000:00:007.3: Host SMBus controller not enabled!

This error is being caused because VMware doesn’t actually provide that level interface for CPU access, but Ubuntu try to load the kernel module anyway.

How to fix it:

  • sudo nano /etc/modprobe.d/blacklist.conf
  • add the line:
    blacklist i2c-piix4
  • reboot

NOTE: for older versions use blacklist i2c_piix4 instead.
NOTE: it works both in VMWare Fusion 5 and 6, and Ubuntu LTS 12.04 and 14.04

#MacOsX : VMware Fusion 5 vs. VMware Fusion 6 vs. VMware Fusion 7 vs. Parallels Desktop 7 [UPDATED]

I will not present functionalities here; it will be just a really brief benchmark.

Configuration:

  • Hard Disk OCZ-AGILITY3 240 GB, firmware 1.15
  • Graphics  NVIDIA GeForce 9400M 256 MB
  • Memory  8 GB 1333 MHz DDR3
  • Processor  2.26 GHz Intel Core 2 Duo
  • MacBook Pro 13-inch, mid 2009
  • Software
    • Hosts: Mac OS X Lion 10.7.4 (Parallels 7, Fusion 5), Mac OS X Mavericks 10.9.4 (Fusion 5, Fusion 6), Mac OS X Yosemite 10.10.4 (Fusion 6, Fusion 7)
    • Guest: Windows 7

Some info:

  • VMs have 2,5 GB RAM allocated
  • VMs have 1 CPU allocated
  • Parallels was tested when disk was new, while Fusion with half of the space occupied
  • Parallels VM has optimize Windows performance on
  • Fusion VM has disk buffering disabled
  • Fusion 5.0.5, Fusion 6.0.4, and Fusion 7.1.2 were tested with Windows Aero OFF
Component Parallels 7.15106 Fusion 5.0.[1-3] Fusion 5.0.5 Fusion 6.0.4 Fusion 7.1.2
Processor: 4.4 4.4 4.4 4.4 4.4
Memory (RAM): 5.5 5.5 5.5 5.5 5.5
Graphics: 5.0 5.0 5.9 5.9 4.7
Gaming graphics: 4.4 4.2 5.1 5.2 4.2
Primary hard disk: 7.2 7.7 7.4 7.4 7.4

NOTE1: Fusion 5 seems to start and stop a bit slower than Parallels 7. Fusion 6 doesn’t improve, but SSD might be slower after this time than it was originally.

NOTE2: disabling Aero drastically improve Graphics performance, so do it!

NOTE3: Fusion 6 improve 3D Graphics performance over Fusion 5, but just slightly, so it is not worth an update. IMHO.

NOTE4: the combination of Mac OS X Yosemite and Fusion 7 has poor graphics performance. It is actually a big regression. I think Apple is to blame here.

#MacOSX : Make your Mac a Wireless Network Bridge

Sometimes you need to extend wireless field; you can do it transforming your Mac as a Wireless Network Bridge.

Why  your Mac? Maybe a new shining Mac Book Pro? Well, let’s say your city has been hit by a big earthquake, so you had to leave your house, but your wireless network still works. Mobile Network won’t  work due to excessive overhead but you can still access internet to contact your friends and family. Sharing your internet connection give this chance also to many other people (this has been happened to me recently).

First you need a Switch or a Router or a Wireless Access Point and Ethernet cables.

Then:

  • Connect your mac to your wireless network
  • Connect Switch / Router / Wireless Access Point to Mac Ethernet port
  • Go to System Preferences -> Sharing
  • Check Internet Sharing
    • Share your connection from: Wi-Fi
    • To computers using: Ethernet and Bluetooth PAN
  • Let DHCP enabled

NOTE: this way you can connect to internet while you can maintain a secure distance from damaged buildings.

#MacOSX : Subversion Server

You know, Mac OS X is a UNIX system based on FreeBSD, so you can do a lot of NERD things out of the box.

This time you will learn how to setup a Subversion Server:

  1. Open Terminal app
  2. Type:
    mkdir -p /Library/Subversion/Repository
    cd /Library/Subversion/Repository
    svnadmin create myproject
  3. Great! Now that you have created the first repository, you need to configure it; type:
    vim /Library/Subversion/Repository/myproject/conf/svnserve.conf

    You’ll see ### commented lines. The lines with the single “#” comment marks are the ones we need to edit. You’ll want to remove the comments (#) and customize these settings to suit your project’s needs:

    • password-db : this specify text file that stores the usernames and passwords of authorized users for your repository. Unless otherwise specified, it will be assumed that this file is stored in your projects “conf” directory. When you create your repository a “passwd” file is created by default.
    • realm : realm tells clients what they are connecting to. It’s recommended you customize this to something like <Project Name> Subversion Repository.
    • anon-access : this directive indicates what anonymous users are allowed to do with your repository. You should set it to none.
    • auth-access : determines what permission level authorized users will have. In almost every case this will be set to write.
  4. Now you must edit password-db file; you can add as many users to your repository(s) as you want. Add at least one user for yourself so that you can begin committing to your repository:
    vim /Library/Subversion/Repository/myproject/conf/passwd
  5. Next you need to change permissions on the Repository directory so that svnserveis permitted to make changes to the files stored there:
    sudo chown -R root:admin /Library/Subversion/Repository
    sudo chmod -R ug+rwX,o= /Library/Subversion/Repository
  6. Start Subversion Server:
    svnserve -d -r /Library/Subversion/Repository
  7. Check if it works:
    cd
    mkdir project
    cd project
    svn checkout --username <user> --password <user_password> svn://localhost/myproject
  8. If it works you should have myproject directory in Repository. Well Done.

NOTE1: /Library/Subversion/Repository is just an appropriate example for Repository, but you can choose every other location.

NOTE2: in svnserve.conf and passwd files remove every white spaces at the beginning of non empty lines or it will not work.

NOTE3: if you want allow users to connect from internet you should set port forwarding on TCP port 3690 (svn default).

NOTE4: check Subversion Complete Reference for more info.

%d bloggers like this: