Oracle - A set of memory structures that manage data files. Every database has at least one instance. An instance can exist without a database. In RAC multiple instances share the same database. Database here is used in the Oracle sense of the word.
SQL Server - The software, memory structures and data files. A computer can have more than one instance. One instance can be designated the default, but secondary instances must be named. Some things are shared between instances.
If you are familiar with what instance means on other databases, please add.