How top chefs develop their signature dishes

What can home cooks learn from the chefs who find inspiration in so much of what they see, from unloved produce to "rubbish" cookbooks?

Oyster ice-cream. Aerated cheese. Medicinal salads. Food that causes diners to exclaim "What the …!" then follow up with "Wow!" How do these dishes make it to our menus?

The answers are myriad. Family histories and treasured cookbooks are common sparks for inspiration, but some dishes spring from long walks and big chats, deep dives into Google, and sketchbook scrawls that lie dormant for years then push to the surface. Connections to particular ingredients often hold the key, not just juicy fruit or vegetables with vim, but also a connection with those who grow them.

We asked some of Australia's most creative chefs where their ideas germinate and how they travel the tricksy path from concept to plate.

Reinventing cheese

"The spark might be an ingredient, a taste, a smell, something I ate in Spain two years ago, even a rubbish recipe book I buy on Amazon," says Martin Benn, whose Sepia restaurant was recently scored 19 out of 20 by chief restaurant critic of the Sydney Morning Herald, Terry Durack. "I put it all in the back of my mind and it will come out in some form or other." He carves out space to create. "I shut the door to the living room, sit with pen and paper, maybe flick through photos on my phone, and give myself time." He also takes long walks with partner Vicki Wild. "We walk and talk about food, wine, ideas, the business," he says. "I don't want to stop creating, it's who I am. I want to explore every avenue I can."

Sometimes I think an idea might be ridiculous. But whatever it is, that idea has its time.

Dan Hunter

When ideas get to the kitchen they usually end in failure, at least initially. "Failure is the good part," says Benn. He reckons 99.9 per cent of his experiments are flops but it's a key part of the creative process. "That's what keeps us wanting to be better every day."

One dish that might have pushed past 99.9 per cent is the Aerated Cheese. "It drove Martin mad for many months," says Wild. Why aerate cheese, anyway? "It does sound crazy," admits Benn. "I wanted to make our cheese course more interesting and playful and, because it's part of a tasting menu, I wanted it to be lighter." He took his idea to the team, "they looked at me like I'm mad", and the process began. After countless blobs of goo and much investigation of temperature, fat and salt levels, a recipe was concocted.

They melt washed-rind cheese, add gelatine, foam it with a cream whipper, pipe it into a ring mould, put it into a vacuum container, then a vacuum chamber, then under pressure and then blast freeze it. (That's the simplified version.) After all that, the success rate is still only 50 per cent. "We have a lot of welsh rarebits for staff meal," says Benn, seemingly happy with the situation. Those that pass muster are topped with apple suspended in sake jelly and a biscuit of roasted buckwheat tea. "Cheese, fruit, biscuit," says Benn, like he is passing over a Salada. "There's our cheese course."

Internet rabbit holes

For chef Thi Le, creating dishes for Melbourne's Anchovy is about finding the tasty interplay between her Vietnamese heritage, fine-dining training and thirst for knowledge, especially about little-known ingredients and cooking methods.

When figs came into season, she pondered what to do with them. Her menu is largely inspired by Vietnamese and Lao food but figs weren't a feature of those cuisines. "Then I stumbled across an internet article which mentioned different figs around the world," she says. "I went down a rabbit hole of fig stories and discovered that one of the oldest varieties came from central Vietnam." Suddenly she was reading – with the help of Google Translate and her mum – about enormous one-kilogram green figs served in a palace dish in imperial Vietnam. It felt like permission to play.

She tried to find such figs in Australia – "not just unripe, but green" – but no one had them. Le wondered what might work in their stead and fixed on green tomatoes. "I flipped it around," she says, "I came up with a salad of green tomatoes and fresh white figs, plus chilli, spearmint, coriander, dried prawns and crispy garlic."

Long walks on the beach

The Iced Oyster at western Victoria's Brae is polarising: puzzling to look at, bracing to eat. An oyster shell is filled with oyster-flavoured ice-cream topped with a green powder of sea lettuce, dried oyster and sherry vinegar. It comes out mid-meal as a weirdly beguiling palate cleanser.

So how did it come about? Chef Dan Hunter puzzled over how to draw three of his obsessions together. Firstly, the beach in cold weather. "I've always enjoyed the Great Ocean Road in winter," he says. "It's powerful and energising." Second, ice-cream. Even in biting wind and sideways rain, Hunter finishes seaside excursions with ice-cream. Thirdly, oysters, and particularly the notion of seducing oyster avoiders. How do you get them to feel the love? "People hate oysters because of the texture, not the flavour," he says. "The flavour is the ocean and everyone loves being by the ocean. What else does everyone love? Ice-cream." Ipso facto, oyster ice-cream.

Hunter sketches ideas in notebooks where they might sit for years. "Then I go back to them – it might be laziness or timing or confidence," he says. "Sometimes I think an idea might be ridiculous. But whatever it is, that idea has its time." The oyster notion lurked for two years but the process in the kitchen was relatively speedy. "I had it clear in my head by the time we started," he says. "We made an ice-cream base with water from the oysters. The first one wasn't good. The second one was. We put it in an oyster shell and away we go."

Transforming memories

O Tama Carey (ex-Berta) has a Sri Lankan mother and has travelled and cooked there throughout her life. Her new restaurant, Lankan Filling Station, will open in Sydney's Surry Hills in June. Carey wants to transport the energy, spice and balance of Sri Lankan street food to Australia, but she's also drawing on her experience cooking Chinese, French and Italian menus, and what she calls "an Australian sensibility, where you go for those cleaner flavours". To develop her new menu, she trawls her sensory memory bank for Sri Lankan elements then teases out her spin on it.

Carey was sure she wanted to use gotu kola, a leafy, bitter herb, also known as pennywort. "The flavour is distinctly Sri Lankan," she says. "They use it a lot, often ground into a dark green paste eaten for its medicinal qualities. People say it's good for diabetes and for the heart."

She also wanted a green salad on the menu, even though salads aren't really part of Sri Lankan cuisine. "There's always been a green salad in the restaurants I've worked in," she says. "The iceberg salad at [French institution] Bistro Moncur was one of the first dishes I learnt to plate, and when I worked at Billy Kwong, Kylie Kwong would always have a herb salad too, even though it's not really Chinese."

Carey took leaves from all those books to create her "medicinal green salad", using gotu kola, betel leaves, parsley, mint and raw onion, dressed with coconut vinegar. "I wanted something crisp and fresh. I also like the way it links to the medicinal backstory that's so important to Sri Lankan food; there's an ayurvedic aspect to the way they put meals together."

The gravity of apples

Sometimes a dish springs from something as simple as engaging with a common ingredient in a new way. Andres Becerra, head chef at Melbourne's MoVida Aqui, had never picked an apple from a tree until a few weeks ago, when he went on a produce-sourcing mission in Victoria's High Country. Wandering the sun-speckled groves at Europa Gully Orchard in Stanley was transformative. "Tasting the apples from the tree was unbelievable," he says. "The aromas and flavours were incredible; you'd bite an apple and see the juice coming out."

Becerra and his chefs tasted six different varieties with farmer Peter Chambeyron. "It was incredible to realise how much effort and work farmers put in and how passionate they are," says Becerra. "Sometimes us chefs in the city disconnect with the countryside, and we miss that connection to the produce." He left with bags of golden delicious, royal gala and granny smith to play with, but also a powerful wish to honour Chambeyron's toil.

On the way home, Becerra mused. "Everyone loves apple pie, right? We asked ourselves how we could do an apple pie in the MoVida style." They came up with a slow-baked apple terrine that celebrates the differences in the three apple varieties. "We experimented with cooking them all – the gala is sweet and falls apart when cooked, granny smith is sweet and tart with a firm texture that's great for baking, and the golden delicious is aromatic and firm too."

A no-waste approach led Becerra to dehydrate the skin and turn it into a powder for garnishing. Also on the plate, biscuity chestnut pastry and barely sweet goat's cheese ice-cream, bringing the flavours of the High Country together on the plate.

Veg gazing

"For me, the biggest inspiration, and the best ideas come from looking, and looking again, at produce," says chef Brent Savage (see video below). Unloved bits of vegetables are a sure way to get his synapses firing, especially when he's creating dishes for his vegetarian kitchen at Sydney's Yellow. He might chew on a root, comb through a plant going to seed, and munch on the bits that are about to be turned into compost. "Often those pieces have more character," he says. "They are more full-flavoured, sometimes so intense they are even offensive, but if you treat them well they can be delicious."

Fennel roots, zucchini stems and broad-bean shoots have all found their way onto recent menus. They're offcuts, but they charmed him once he took time to get to know them. Listen hard "and they almost tell you what to do with them", he says. "Odd bits get the creative juices going. They're not obvious. You have to use your brain, go through that process of trial and error, which is what chefs love."

Of course, you can't tick "fennel root" on a supplier's list. "You have to get to the farm and think about the plant from root to tip," he says. Savage works closely with Carmel and John Zavaglia, farmers in Glenorie, a semi-rural suburb 44 kilometres north-west of Sydney. "You look at the stem, root, leaves – that's how I started using the broad-bean shoots. They were just mulching them into the ground and now they're one of my favourite ingredients to work with."

Savage sautes the leafy broad-bean shoots, dresses them with sherry vinegar and "beetroot oil" (left over from slow-roasting beetroot), and serves them with spiced toasted oats. "They're like broad beans, but with a nutty flavour and a texture you can get a few chews out of," he says. "They're super tasty."

Tips for creative home cooks

What about the rest of us, those of us without notebooks full of ideas, farmers to visit, and brigades to clean up our messy experiments? Is there anything we can glean from these chefs and their creative process?

■ Carey suggests choosing one ingredient and bouncing off that. "Think about what it tastes like, what it's like cooked, raw, how you might cut it," she says. "Mandarin is a good one. It's citrus so you might think of lemon and think, I love roasted fish with lemon. Mandarin is sweeter, so maybe you want something bitter, so you reach for radicchio. Then raw radicchio is very strong, so maybe you decide to cook it because that will tone it down a bit."

■ Get the best, freshest produce and let it shine with simple preparations, says Becerra. "If you have great produce you don't need to complicate it."

■ Embrace your mistakes, says Savage, because they can be opportunities to get creative. Let's say you burn onions. "Instead of throwing them away, blend them up and turn them into a dressing and see whether that brings the sweetness out."

■ Benn suggests home cooks read recipes all the way through then consider what could be changed. "What would make it more to your taste? Maybe you don't like sardines but you like anchovies. Change it. There are no rules anymore."

■ Experiment with different stocks, says Hunter. "People boil or steam vegetables but you could just as easily make a flavoured water with dashi or dried shiitake and cook brassicas in that. Simmer the stock, don't boil it – think of it like a tea. You'd never boil a teabag. At the end, liven it with citrus, but rather than a squeeze of lemon juice, use the very outside of the zest. You smell it as well as taste it and bring in more of the senses."

■ Le encourages home cooks to strip ingredients back to their flavour profiles, for example salted ricotta always reminded her of funky fermented bean curd. "At Cumulus Inc we used to do green beans with salted ricotta. When we did zucchinis at Anchovy, I came back to it. We did a mayo base and folded fermented bean curd through it."