A valid order must have a non-zero amount, and also amount x price must be non-zero. In other words, an order that would trade something for nothing will be rejected. Keep in mind that these values are rounded to the precision of the coin, so a very small amount may cause amount x price to be zero when rounded to the coin precision, even when it is non-zero before rounding.
Also, a valid order must generate non-zero trade fees. In SouthXchange the trade fee is a percentage of the amount that is received in that trade, and that amount has to be non-zero (also when rounded). This condition is not enforced when trading all of your balance, so if you have a small amount that you wish to sell and it would generate zero trading fees then you will be able to trade as long as you sell everything. If you try to sell a partial amount then you will have to make sure the trading fees are non-zero.