I don't think we can say the Greeks were exploring complex numbers. There's something about Diophantus finding a way to combine two right-angled triangles to produce a third triangle whose hypotenuse is the product of the hypotenuses of the first two triangles. He finds an identity that's equivalent to complex multiplication, but this is because complex multiplication has a straighforward geometric interpretation in the plane that corresponds to this way of combining triangles.

There's a nice (brief) discussion in section 20.2 of Stillwell's Mathematics and its History