Oblivion uses a heavily modified Gamebryo engine, the successor of the NetImmerse Engine used in Morrowind. It is much more efficient than its predecessor and features -- among other things -- advanced pixel and vertex shaders.
Oblivion uses SpeedTree RT to render realistic-looking trees and foliage.
AI
Oblivion uses Bethesda Softworks' own Radiant AI engine. It is based not on specifying exact actions (scripting) for every NPC, but rather goals that are defined for every NPC which they must try to achieve in accord with the current state of the gaming world. If an NPC is hungry, they will try to acquire food; first, they will try to buy food, and if that doesn't work, they may go and hunt deer. If the moral ranking of the character is low, he may attempt to steal food from a merchant.
Combat
Bethesda has implemented and tested three different combat systems to see which one worked best in terms of gameplay and fun.
Stealth
The components of stealth have been significantly overhauled. Shadows affect your visibility, and your current visibility is shown on your crosshair.
Magic
Spells work fairly similarly to Morrowind. However, the Magicka regeneration has been much improved and spells can be cast without switching to a casting stance.
Alchemy
Potions can be used as a poison on weapons (including arrows) or drunk by the player to enhance their abilities.