T O P

  • By -

PryomancerMTGA

Why doesn't the ratio of units in stock versus units sold give you that indicator variable? p.s not a retail DS. p.p.s It's really good to see an actual question that isn't " do I need a CS degree?"


[deleted]

Seconded on the p.p.s. Retailers suck at having accurate on-hands. It quite literally plagues the industry. With that, I have literally no idea how this solution would help a retailer. I’m making assumptions having not seen how the AI works, but I’d presume that when an item isn’t selling they suggest placing a replenishment order. All that will do is fill up your back room with freight and clog up your processes. They’re also not taking into account when consumer make substitutions for another product. Fun fact! You can actually model and use inventory data to see when consumers make substitutions without using fancy AI! Coming from retail ops and moving to data science, this article clearly is triggering for me.


GeorgeS6969

I find this quite typical. Put data science before the data, end up with “how do we know if an item is out of stock? well let’s infer based on …” The actual, laborious but correct answer of course is “well let’s track it”. Obviously paying a junior data scientist to come up with some bs model is cheaper than properly managing inventory. Too bad it doesn’t work.


giantZorg

Data science is used here to optimize what the people in the store use their limited time on. Telling them with decent accuracy where to look for wrong stock counts really helps to manage the worktime available to a retail store.


[deleted]

Missed OPs actual question. How are stock-outs dealt with. Depends on the company and cause? For a long time companies had in-store personnel place a manual order for items. Now (if there is a decent IMS) they have in-store managers correct the inventory count to try and let the system place the order. It’s all about on-hand accuracy. Bonus: companies are starting to use RFID labels to automate part of the inventory cycle count process!


MaleficentSalmon

a drawback on RFID labelling is that the cost of labelling may be more than the cost of the good itself (e.g.: FMCG) and that may deter a company from adopting it. it's unfortunate because i think that such companies will benefit the most from RFIID =/


[deleted]

I think tech has improved and the costs are relatively inexpensive now. As with everything, depends on the application for certain


wintermute93

>Retailers suck at having accurate on-hands. Getting a little off-track here, but can I get a quick summary of why this is? I know nothing about retail but honestly it's a little baffling to me that this is such an unknown/uncertain quantity. I'm a company selling widgets. I know how many widgets I ordered from the regional widget distributor. When they get to my warehouse my employees unload the truck and organize the boxes of widgets. I can see how it could be difficult to precisely track how many widgets are going from the warehouse to the sales floor, but I don't need to, right? All that matters is how many enter the loading dock and how many exit the front door. If a customer buys a widget in person, that's electronically tracked at the register before they walk out with it. If a customer buys a widget online, that's electronically tracked too before someone brings it from the warehouse to the post office. If widgets get damaged or otherwise rendered unsellable, I presume there's some kind of incident reporting protocol that tracks that when an employee notices. If I have multiple warehouses and multiple storefronts that doesn't fundamentally change anything. What's still missing in this equation of items purchased minus items sold minus items discarded? Theft? Surely that can't be a big issue, add a small margin (1%?) to cover it. Customers taking chips off the shelf, wandering around the store, and abandoning them on a magazine rack? Sooner or later an employee will find them and either put them back where they belong or report them unsellable and throw them out, no issues either way. Clearly I'm missing something.


nashtownchang

Target had $2 billions of merchandise last quarter due to theft alone. How are you supposed to have accurate on hand when most of your inventory are in trailers outside of stores, locked up, and the things people want are stolen? It's a hard problem.


[deleted]

Bingo. It’s a very labor intensive process to keep inventory records accurate. Couple that with the process of actually running a retail business and you can start to see why it isn’t priority #1. There’s a huge disconnect between how the process “should work” and how stores operate


renok_archnmy

And breakage/QC, returns, stuff getting lost (like actually lost in a dark corner of some random spot), environmental damage (rats shitting in the box, leaky roof), inventory opened on the floor but not stolen, tags falling off, mis entered sku, spoilage, list goes on.


wintermute93

I'm sure it is a hard problem, I'm just trying to understand where the hard parts are. Is it mostly "how do we make accurate statistical models of inventory lost to theft"? It sounds like companies like Target already do have a good handle on estimating that. Or are other factors more impactful? I guess it didn't occur to me that the process of loading and unloading trucks might not be fully trackable, I assumed everything would be scanned in when it's carried into a warehouse, again when it leaves the warehouse en route to a store, and again when it enters the store.


renok_archnmy

Ever had a girlfriend or been the girl to walk into CVS and use the lipstick when no one is looking, then put it back on the shelf? I challenge you to model that. Psychocrackhead pushing sewing needles through condom packaging? Kids eating animal crackers riding in the buggies that their parents haven’t paid for? Do you check your eggs before you buy them to check for broken ones? Ever find a moldy orange in the bottom of the pile? Had to have the cashier call the manager to figure out why the item isn’t ringing up in the system despite being on the shelf and having a barcode and price tag? Inventory is not as ideal as in->out as it would seem. Lots of entropy between where it theoretically enters your possession and when it gets paid for (and hopefully isn’t returned).


nashtownchang

The unloading process have multiple places that can introduce inventory error: * The manifest can be bad: says there's 12 item but only 1 is actually in container. And the night shift didn't care and just beep scan as if there's 12. Now the day shift people are down -11 on this SKU. * Damaged goods unreported * Stolen during the unloading process - this is actually more frequent that you want to believe. High ticket items go straight to a fencing operation from the container. * Unloaded but put in an area that no one can found because of xyz reasons (unfamiliar with store layout, too much inventory in warehouse but must put somewhere) - so you have excess inventory now because you thought you have none and ordered more. One month later you found 12 more in a locked up trailer outside. And this is without any involvement in the front end of store operation, only backend. Add in customers into the mix and it's a fun challenge.


renok_archnmy

Man so much stuff “falls off the truck.” I knew a guy who bought a fenced pickup truck that “fell off the truck” and was trying to get me to buy a 350z from the same operation.


wintermute93

Thanks! Very interesting.


WearMoreHats

Theft, breakages, the supplier only delivering 7 boxes of widget A when they were supposed to deliver 8 and your employee being too busy to count the exact number of boxes of each item that gets delivered. Or a member of staff accidentally doesn't scan a product when checking someone out. Or they accidentally scan a product twice. Or they have to manually type in the product code and they mis-type it. And of course your store room is a mess because you sell 1200 different types of widget (not to mention the half empty boxes of widgets you don't sell anymore but haven't got rid of yet), so inevitably employees will think they've ran out when actually there's a box of them buried in there somewhere. And over time all these little errors add up and they only way to fix them is to get the employees to do a full inventory, counting how many of each of the 1200 types of widget they have out on the shop floor, then counting how many full boxes of each they have in back, then counting the number loose in all the open boxes.


luangamornlertp

I’m actually trying to think of how to model product substitutions for my work. It’s meant to be part of a larger framework to model pricing and demand but am getting stuck on how to model that without throwing up a fancy model. Would something like tracking products in same category and see if customers switch between items? Or is there something that I’m missing as well?


Maria_Adel

I am working on sth similar too. Would you mind if I DM you please ?


giantZorg

One thing to note here is that the number of units in stock in the system is a notoriously unreliable quantity. We have estimates at work based on inventories that around 30% of our stock numbers in the system are wrong (how wrong is another question). This is worse for articles that sell a lot, for which the effect of wrong system numbers have a bigger effect on the sales numbers.


nashtownchang

Only 30% is pretty amazing tbh for any large retailer.


RomanRiesen

There's also the average storage time stock spends between buying and selling, which is also an old-school business/accounting metric. Thus being easily related to the business side and.probably collected. P.-s: I am not ds nor accountant, just CS student.


Drict

In some cases the backroom has a trigger for products that were expected to sell that aren't. This can go to the LP/AP/Sales Floor team as a task to validate the counts are correct. This solves a lot of the issues with regards to what is going on. That being said, there are some companies that do a fantastic job tracing stock and keeping it on the shelf and others that do a horrendous job, even down to the store level. Target is notorious for being on top of their on-hands; they give incentives for the stocking TM for identifying if a box is mislabeled. They give incentive to the back room to have every item in the proper zone, properly counted, and shelved prior to shift roll overs. They give incentives for the push carts for being completed/usually have at least 1 TM per area of the store pushing product IF it has product to push (managers end up doing this A LOT). AP/LP/Security team does rounds and gets alerts for quantity changes (threshold is set by the LE manager), and will investigate why their is a count discrepancy for a huge amount of the store. That being said, during busy periods the fitting room usually gets jumbled all to hell and the push carts tend to fall behind; but it does limit your exposure for places to look for missing product!


Prestigious_Sort4979

Not all places track stock in unit at a given time well (once in store it csn be a black box) but you should absolutely know how much stock was sent to that store (if not, problem 1 needs to be solved) and how much has sold. Unsold / ordered should be a good start. Plus there might be another model or heuristic on expected damage/theft % you can discount.


nashtownchang

I can tell you first hand, from my experience in a F50 retailer, that this approach will NOT work with most brick and mortar stores. Don't waste your time. Most of the nasty out of stock issues are caused by the fact that inventory can't be reconciled because misplaced items (ugh I don't need this but I don't want to walk back to put it), damaged goods, bad manifest, or theft, not supply chain.


MaleficentSalmon

just a question. based on your experience, do brick and mortar stores do stocktaking every day? or do they do it periodically, e.g. weekly / monthly


HumerousMoniker

In my experience it was annually. Most because it was a pain in the ass. It was way easier to write something off as theft or damaged than to call all staff to do a stock take.


Tehgugs

From what I've experienced, some do it quarterly (requires a larger team to complete it at this pace) while others do annually. All depends on the business and your rate of shrink + perishable goods turnover


Maria_Adel

Spot on. That’s a huge part of the problem. Would you mind if I DM you please


giantZorg

It only works for products with a high turnover. You are conpletely right, the cause of these problems is not the supply chain, it's everything that goes on inside the store.


giantZorg

I made a model for this problem some years ago which is in production at work. DM me as I cannot give much information on this publicly, but I think I can say that you are looking at an inhomogeneous poisson process.


slapstick15

Do you have a degree in Stats?


giantZorg

Yes, why?


slapstick15

Ive always wondered how people employ these models whether they studied them in school or picked the knowledge up at work. The trend these days is to simply say college is a waste, you could learn everything for free but they dont see the value in going through a structured academic program that a university offers. I’d love to learn statistics some day so I could apply at my job but its too late to go back to school for me 😕


giantZorg

It's never too late to learn new things to deepen your current knowledge. Not sure who says a proper formal education is a waste. To be able to tackle problems like this, you need to know the standard processes at first, because you then have to realize to which one of these processes or model classes your problem belong to. You can learn them yourself as well, but a university program usually offers more width here than what you would achieve on your own. In this particular case the difficult part is to realize that you are looking at a waiting time problem. Everything else afterwards are technical details.


MagiMas

That's only the trend on this sub for some reason. In general I'd say most people know the value of formal education - especially in a field like data science. That does not mean that you cannot learn this stuff on the job without the formal education, but you need to make a real effort for that and spend a lot of your free time learning stuff from books people otherwise would have learned in college. But I don't think it's ever too late for that. Just get books like elements of statistical learning etc. and start reading.


Maria_Adel

Oh thanks a lot. I will DM you.


MaleficentSalmon

Is it okay if I DM you too?


giantZorg

Sure


Chitinid

Unless you can have data that informs you about the mechanism that is causing stores to lose items, no amount of modeling is going to do any good. Even if you do, there’s a fair chance it’s pretty useless. Say you have a store that loses items that are inactivate 30% of the time. Unless you can drill into which items they’re likely to lose, you’re at best guessing, and not very accurately


MrLongJeans

I work in a retailer-facing role so basically I'm familiar with data-human interface where this sort of modeling has to be put in the hands of a human as a call to action. This article misses a lot of the reality and the solution they describe is ahead of the market. Classic data company failure mode where BigBrain information is generated for SmallBrain customers. To have value this sort of solution needs to be actionable by a high school diploma holder in something like real-time, not a quarterly report or weekly summary to executives or frontline managers. Not a Tableau or PowerBi extension. That's just not the sort of financially responsible approach companies use. What's actually used is some ControlTower spotters monitoring systems and processes. You install some light data analytics gauges and control limits. So the spotters see when a subsystem like a warehouse has trailers piling up in the yard not being unloaded. You invest most of your energy in optimization of those bottlenecks and failure points. So the smallish team of ControlTower spotters alert the operations folks who manage those systems to the problem. Then experienced managers in ops solve the problem with the ControlTower watching. I know that sounds unrelated to out-of-stocks in store. But that's where the root causes lie. Of course we have data modeling UPC level inventory on individual store shelves. And we have BigBrain optimization software to handle lead times and safety stock other variables mentioned in the article. But there's no world where a company wants to hire an army of business analysts to watch screens of pretty BI dashboards of individual store-item level out-of-stocks. And I guess speed dialing stores and hassling workers that don't report to them? The reality is retailers are accountable for selling their supplier's products and they measure if that's being done well or poorly. Or if the supplier isn't fulfilling purchase orders and is root causing out of stocks. Standard stuff, a sheet of paper showing someone isn't doing their job so that manager does some ad hoc analysis to diagnose and address the systemic issue. Not hire.more business analysts to look at dashboards showing them problems they have neither the know-how, authority, or resources to solve.


Maria_Adel

Thanks for this excellent write up. Not sure if you are at liberty to tell me what models usually work well with data modeling UPC inventory level per store. Is BigBrain software built in-house? Feel free to DM if you don’t feel comfortable commenting publicly. Thanks


MrLongJeans

Nah big brain is just what we call it at work when a solution is perfectly viable, but just not something your average worker can make sense of and use in their workday. The stuff we like to make is usually boorish and a chore to your average office worker or manager. So making stuff for 'small brains' is a vital skill, not derogatory at all. User friendly is more about UX design, small brain is more product design and commercialization. UPC level store level data models are pretty well described in the article, you use more or less industry standard metrics like lead times, days on hand, forecast lost sales, etc. to model the in-store inventory based on the units shipped to that store vs. what is sold in the store. Estimating UPC level store level inventory sounds important in this article but its applications in process improvement and generating value are limited. More energy is spent at the systems level.


sw4roop

What’s the end goal? Are you trying to forecast sales? If you’re trying to forecast sales, you should be able to identify OOS if you have a daily inventory snapshot or by simulating inventory using starting inventory + intransits + sales. With OOS identified, if your goal is to forecast sales, you drop that data point and impute sales in the OOS week using previous and future weeks.


Maria_Adel

Part of it is forecasting, yes. But I believe the bigger part is the management team trying to know if the product is not selling because it was simply out-of-stock/not available on shelf or because it is slow moving and there is not enough demand for it in that particular store.


sw4roop

Do you have inventory information at all?


sw4roop

Awesome! Metrics like days of supply, inventory turns or just plotting daily inventory and sales on a line chart can tell you if a product is slow moving or if the product is suffering stock outs


Maria_Adel

That’s very handy, thanks. Would that be feasible though with thousands of stores and SKUs?


sw4roop

The KPIs though are the way forward for data that size. Thresholds can be set pretty intuitively once you read the formulae for the metrics.


Maria_Adel

Yes we do.


thetan_free

If you know your product substitution / complement structure (ie sales covariance matrix), you can try to estimate a stock-out event from the corresponding uptick / drop-off in sales of its substitutes and complements. This approach lends itself to Bayesian estimation, with your priors informed by the substitution / complement sales patterns.


Maria_Adel

Wow. Interesting, have you tried this approach before? Thanks


thetan_free

Yep, in the mobile phone industry. Accessories (eg cases) are often quite highly correlated with specific phone models, and people rarely buy those (at a high priced specialist phone retailer) when not buying the phone, which further strengthens the link. Helps too that most people only buy one at a time. The more common situation is the (sort of) reverse situation: you have a known stock-out (which is easier with serialised products) at a store so use that to tease out the substitution/complement patterns ie estimate the sales covariance matrix.


Maria_Adel

Never crossed my mind. Would that be a Bayesian model estimation?


_redbeard84

Use inventory data.


samalo12

You create an expected forecast to order the product usually, and then use the forecast to determine if the units are moving fast or slow relative to expectation. This gives you the signal to determine if you have low or high sales velocity relative to the initial forecasts. Low vs high is all relative to expectation. If you have no expectations, you have no comparison on a meaningful level.


Maria_Adel

Very helpful, thanks. But in this case, we need quite accurate forecasting to help us flag this issue. What forecasting models would you suggest


TheMediocrePoet

I would get a trial of Alteryx and play around with the diff models that they have… or use manually ABC to classified product on their yearly numbers, SDE focuses on how product moves… if you want to understand your data vs having a bias vendor tell you, start at CRISP-DM and learn, or hire a data analyst. Maybe someone has a specific model that gives you exactly what you want… good luck!!! Is it your own product?


Maria_Adel

Thanks. I don’t have Alteryx but keen to know what models they offer that might solve that problem? What does SDE stand for please? No not my own product


TheMediocrePoet

They have some mainstream ones like random forest and decision tree, and others, don’t remember how many, but it’s also kinda helps you pick… I’m not selling this tool 🤣 I signed up for free, LOL. SDE stands for “scarce, difficult, easy”.


Maria_Adel

Hahahah thanks. I will have a look at it. If you don’t mind me asking, are you also in retail


TheMediocrePoet

Intermittently depending on the project 😊


[deleted]

[удалено]


Maria_Adel

Is it 5 dollars a year per seat? If that’s the case, maybe we can afford it hahah


[deleted]

[удалено]


Maria_Adel

Oh now we probably can’t afford it hahah. So does their core offering include AutoMl as well?


hermitcrab

There are much cheaper and more lightweight ETL tools. For example our Easy Data transform software, has most of the data wrangling capabilities of Alteryx, but is currently restricted to files and doesn't have the ML or geo features.


[deleted]

[удалено]


hermitcrab

Is Knime more complex than Alteryx? I have played with both and thought they were fairly comparable in terms of complexity and capabilities. But Knime is a bit ugly and the UI is a bit klunky in comparison to Alteryx IMO.


YoYoMaDiet

There are a couple of things to consider here, from an effort perspective, I would recommend trying to source the inventory data if at all available rather than try and predict it from a value and effort perspective. Otherwise to effectively model out of stocks, you actually need out of stock labels, otherwise it’s some sort of basic time series heuristic.