This is a question I get asked from time to time. The short answer is no, you should not. Although grout does not add to the stability of the tile installation (unless it is epoxy grout), you still need to grout it.
Why you need grout
A lot of natural stones, namely granites and marbles, are manufactured to be consistently sized. For the most part all the tiles are identical. This makes a lot of people want to install them without grout lines. Although in some people’s opinion butting the tiles against one another looks better than having even the smallest grout lines, it is not a recommended installation procedure.
Even if all the tiles look like they are the same size I can nearly guarantee they are not. Unless they are “rectified” they will differ, even if only a tiny amount, from tile to tile. Attempting to butt the tiles will result in a “jog” of the lines between them. The larger the area, the more those lines will run off. By leaving even 1/32 of an inch grout line you will be able to compensate for the difference in tile widths.
You also need grout to ensure that nothing can get between your tiles. Look at it this way: would you rather have a very small grout line filled with grout or a very, very small grout line filled with spaghetti sauce? No matter how tightly you attempt to butt the tiles, there will still be the tiniest space between them. Not grouting them leaves open the possibility of all types of unruly things filling them. Then you have to clean them out risking the possibility of damaging one of the tiles.
The final reason I’ll throw out there is that no matter what substrate you are using there will always be movement. Always. Placing the tiles against each other will eventually damage them. If you continuously rub the edges of two tiles together one or both will eventually chip (and you need to get out of the house more, or at least find another hobby). The expansion and contraction of wood or concrete will do the same thing. Although you can minimize this using different underlayment materials, it will still move.
I hate grout, I really do. If it were up to me I would install most tile and all granite and marble with no grout lines at all. I can’t do it. Even though it will look better initially, eventually it will ruin the tile. The best thing to do is use the smallest grout line your particular tile will allow and get a grout that closely matches the tile. For most granite and marble tile I install I use either 1/32 or 1/16 inch grout lines. In most other tile I will use 1/16 or 1/8 inch lines. I try to use the smallest grout lines the tile will allow.
To figure out how small you can go, place nine tiles in a 3 X 3 foot square butted against each other. Measure corner to corner diagonally both ways and see how close they are. If they are within 1/16 inch that is the size grout line you can use safely.
Please resist the temptation to install your tile without grout. Grout sucks, believe me, I know. By choosing a matching grout, though, you’ll be happier in the end and your tile will last significantly longer.