Al Qaeda leader Ayman al-Zawahri dismissed Islamic State and its leader Abu Bakr al-Baghdadi as illegitimate but said his followers would join them in fighting the Western-led coalition in Iraq and Syria if possible.
In an audiotape on the internet, Zawahri said: "We don't recognize this caliphate."
It was not clear when the recording was made but references to events suggest it was made at least eight months ago.
Islamic State has declared a caliphate in large parts of Iraq and Syria under its control and sought to expand its power in the Middle East, becoming a greater security threat than al-Qaida in the strategic oil-producing region.
Despite rivalries between the two Islamist organizations, former Egyptian doctor Zawahri suggested there was still room for cooperation when it came to combating the West.
"Despite the big mistakes (of Islamic State), if I were in Iraq or Syria I would cooperate with them in killing the crusaders and secularists and Shi'ites even though I don't recognize the legitimacy of their state, because the matter is bigger than that," he said.
Zawahri did not elaborate, but his comments raised the possibility he could seek cooperation with a movement that has in many ways eclipsed al Qaeda, its predecessor in Iraq.
"Zawahri says that ISIS is absolutely not a caliphate and Baghdadi doesn't deserve to be caliph," said al-Qaida expert Will McCants of the Brookings Institution. "However, it is an emirate and if he were in Iraq and Syria, he would fight alongside it. That seems like a bit of an olive branch."
Any cooperation would complicate efforts to stabilize the Middle East, where militant groups have gained influence and escalated attacks since the Arab uprisings of 2011 toppled autocrats who had contained them.
While al-Qaida has specialized in high profile bombings, Islamic State seems bent on seizing and holding territory in its quest to create a self-sustaining caliphate.
Aside from strongholds in Iraq and Syria, it gained a presence in Libya and its followers are staging an insurgency that has killed hundreds of soldiers and police in Egypt, the most populous Arab country.
Islamic State's affiliate in Egypt released photographs on Wednesday of its military commanders teaching masked militants in military fatigues how to fire rockets at tanks and planes.