Turing-complete
languages or
machines are those
that can
express arbitrary
computations, that is, they can emulate arbitrary
Turing machines.
This is not at all the same as the capability to solve any mathematical problem: many perfectly defined mathematical problems are provably unsolvable.
As a matter of fact, the Turing machine was invented to prove this.
A simple Turing-complete formalism is, obviously, the Turing machine; another, the lambda calculus; another is the arbitrary rewrite grammar often used as the last formalism in the Chomsky hierarchy.
Very small subsets of most programming languages are already Turing-complete, on the condition that they can access arbitrary amounts of memory. C, for instance, isn't, because it must address all memory with pointers of a fixed size; but a variant that uses pointers of arbitrary size would be trivial to define.