Around here landscapers are a dime a dozen. However, there is only a handful of contractors that have the experience and knowledge to do hardscaping, ie retaining walls, birck walkways, stone patios, etc. It is a very specialized market, but if you have the tools and knowledge, a very good business. It may be a specialized segment of the market, but you can expand your business into excavation, landscaping, and foundation work if you choose.
My best recommendation is to go to school. I have a bachelor's degree in construction engineering and management. It has gotten me farther in this industry than any of my past experience. Companies are looking for educated people to run jobs, and manage projects. It is easy to go to the hall and get the labor, but much more difficult to find the management for the project.