TOKYO  — Shelters made of cardboard start popping up in the basement of Tokyo’s Shinjuku train station right before the shutters come down at 11 p.m., in corridors where “salarymen” rushing home and couples on late-night dates have just passed by.

Dozens of homeless people sleeping rough in such spots worry that with Japan’s image at stake authorities will force them to move ahead of the Olympics. Already, security officials have warned them they will likely have to find less visible locations by the end of March.

The former laborers, clerical workers and others sleeping in cardboard boxes are a not-quite-invisible glimpse of a more pervasive but largely hidden underclass of poor in Japan, a wealthy nation seen as orderly and middle class.

Efforts to clean up what some see as urban blight have preceded every recent Olympics, including those in Beijing, London and Rio de Janeiro.

Tokyo city officials deny they are moving to force the homeless out specially for the Olympics. They say trying to get them into shelters is part of an overall welfare effort to get them off the streets and find them jobs and housing.

“There is nothing more than the programs we already have in place to help the homeless,” said Emi Yaginuma, a Tokyo city official in charge of such programs.

“We keep trying by making the rounds and talking to them, but all we can do is to try to persuade them.”

In theory, overnight sleeping at train stations is trespassing. In practice, the homeless have long slept in Shinjuku station and other spots. JR East, a major train company servicing Tokyo, doesn’t have regulations on the homeless and employees handle situations as they come up, such as passenger complaints.

Just as the homeless arrive for the night, a public speaker overhead is warning that sleeping in the station isn’t allowed.

As preparations for the Olympics began years ago, homeless people camping in a park in Tokyo’s Shibuya were forced out to make way for development and a soup kitchen program there was moved to another, less visible park nearby. Advocates for the homeless fear that was just the start.

Homeless people were evicted in 2016 from a park near where the New National Stadium was built, the main arena for the Olympics.

Like the U.S., Japan has a relatively high poverty rate for a wealthy nation. It also is less generous with social welfare than countries in Europe, and lacks the sorts of private charities prevalent in the U.S.

Nearly 16% of Japanese fall below the poverty rate, with annual income below the cutoff of 1.2 million yen ($11,000), according to 2017 Japanese government data. The poverty rate for single-adult households with children is way higher, at 51%.

The unraveling of extended family support networks and job insecurity have left many in Japan vulnerable to setbacks that can lead to homelessness. Japan’s culture of conformity leaves many, including families, ashamed to seek help.

Most of the homeless sleeping underground in Shinjuku, a glitzy shopping area fringed by red-light districts, high-rise offices and parks, are older men.

Shigeyoshi Tozawa has a lacquer begging bowl with a few coins, three tiny, solar-powered toy figures with bobbing heads bought at a 100-yen ($1) store, and various bags filled with blankets, clothes and other items, including his poems.

“Last night/ dream of a future trip/ it is dark,” goes one poem. Passersby sometimes give him money for the poems, he says.

“This is my community. We all help each other,” Tozawa said. “There are no dirty homeless here. We are all ‘trendy.’”

In what’s clearly a routine, he and the others quietly prepare for the night, picking their favorite spots, neatly folding blankets. Some change into sleepwear and wipe their feet clean with wet towels, daintily placing their shoes beside their lopsided cardboard shelters.

Tozawa and the others are relatively well-dressed, in handout down jackets, baseball caps and camouflage sweatpants. Some have cell phones and other gadgets. Many have some money in the bank. They get by making the rounds of downtown soup kitchens run by church and volunteer charities, and other spots where they can get free rice balls or sandwiches.

Many of those sleeping rough are “working poor,” said Daisaku Seto, who works for a nonprofit for refugees and a consumers’ food cooperative called Palsystem. He says some suffer psychological trauma and need training to get better-paying jobs. Once they drop into poverty, they rarely find their way back out.

“We need to come up with ways to help that empower them,” said Seto, who is a leader in a one of the leaders of a grassroots group called the Anti-Poverty Network.

Yukio Takazawa, executive director of a support group for the poor in Yokohama’s Kotobukicho, an area of flophouses where homeless people also tend to congregate, worries the worst is to come.

The construction boom from the Olympics will be winding down, reducing chances for odd jobs for day laborers. The younger poor, who now spend nights in Internet cafes, likely will eventually end up on the streets, said Takazawa, who has been working with the poor for 30 years.

Finding affordable housing in Tokyo is tough. Rents are high and landlords tend to be finicky. Just getting a rental contract can require six months of rent or more up front.

Those unable or unwilling to get apartments camp along river banks, in parks and train stations. Welfare offices try to get people to move into shelters but many, like former construction worker Masanori Ito, resist. “They have rules,” he said, munching on sandwiches he got from a volunteer.

If he has to move, Ito said he plans to find some other warm outdoor spot.

“I don’t know where we will all move next,” he said.