IBM Rational ClearQuest ActiveX Buffer Overflow

May 25, 2012

IBM Rational ClearQuest is a management system that monitors and manages comprehensive software changes in the system. It provides change tracking, process automation, reporting and lifecycle traceability for better visibility and control of the software development lifecycle. It supports multiple operating systems, including AIX, HP-UX, Linux, Solaris (Sun microsystems), Windows family.

The IBM Rational ClearQuest supports the connection to various data sources, for example, the Rational ClearQuest database. The CQOle ActiveX control in cqole.dll in IBM Rational ClearQuest can be used to connect to the database by a web user. The CQOle ActiveX control provides multiple API methods for users to contact the database.

A heap-based buffer overflow is found in the Ole API in the CQOle ActiveX control in cqole.dll in the vulnerable versions of IBM Rational ClearQuest. The vulnerability is due to an insufficient validation of the user supplied data, which may overwrite a specific content in the memory.

Dell SonicWALL UTM team has researched this vulnerability and created the following IPS signature to detect the attacks addressing this issue.

  • 7849 IBM Rational ClearQuest CQOle ActiveX Heap Buffer Overflow

This vulnerability is referred by CVE as CVE-2012-0708.