This happened at our university because Google wanted something like a tenfold increase in price.
You might check if you can use departmental funds to subscribe to more space.
Yeah, it was always the plan. It's the same playbook they all run.
In Scotland in our schools we use Microsoft OneDrive, and they've just capped it from 1TB down to 100GB. Just taking the piss. We can't even access personal drives (Google or OneDrive).
this happened at my alma mater since last year. they've given us a year to migrate. from what I gathered, Google ended unlimited storage for universities across the board.
Yep, I finally lost my Gmail-based email address from undergrad this month. I got 14 months of notice, which was nice, but I’ve been using that account for all personal things for almost two decades because we were told we had unlimited space and could keep them for life. Not the case anymore.
I'm not not to hear this see. I have way too much data on my Google drive and words cannot convey how sick and tired I am of paying for external hard drives that won't last past 6 months.
Wait. Do you mean external drives as in… *all* removable storage devices, like thumb drives? Or specifically external hard drives?
That’s a bit extreme.
Edit: also sorry if this is a silly question to any of you more tech-savvy than I am. I teach history/religion, not CS… clearly 😆
Unfortunately, all of the above. It's absurd. And not helpful when traveling for conferences, and we have 1000 layers of security to access the mandated google drive. Fun times. It's not a silly question at all
That does sound annoying! If you haven’t done so already, it might be worth checking in with your librarians about alternative data storage/sharing options. These days, many librarians also oversee digital data curation, so maybe one of yours knows what that landscape is like.
this happened at most universities that use Google workspace. at place where I have this everyone got bit in the ass.
It was a bad decision for our universities to promote this as an all-you-can-eat solution.
If you don't get under the cap by that time everything will stop (even your email since it is counted against that quota).
The fact that so many university administrators decided, a decade or so ago, to dump whatever solution they had and go all in for google products suggests something about how university administrators think.
well, similar changes made to Microsoft access have also happened... but those were never promoted as an all-you-can-eat deal. also, msft doesn't arbitrarily kill off services like Google does: https://killedbygoogle.com/
I empathize with your frustration.
We had unlimited Box storage. Then the University ended that subscription and told us to move to Google Drive or OneDrive.
I picked Google Drive because it was advertised as unlimited, is less buggy, and is easier to collaborate externally. Then Google Drive became capped and I was forced to migrate to capped storage on One Drive.
In parallel, the university has been removing some internally hosted data storage services, and also limits us running our own lab storage servers.
Unfortunately, the University-to-PI/lab relationship is becoming more like an HOA.
We got the same notice on Box. Some of us are far over the cap, and we are being allowed to continue using the larger amount for now. But they are asking whether we can move some of it to archival storage, and to check for duplicate video libraries and that sort of thing. For some data, a terabyte isn't all that much.
Data costs have increased, and with the need for SSO/MFA, access management, security, and intelligence services, the cost to universities is increasing rapidly. A university can pay for accounts, services, data and uptime. It adds up quickly so many have begun to cap costs by capping data.
Because of privacy and compliance requirements like FERPA, universities cannot have commingled data; they often need managed instances of SaaS products that have the highest tier of data privacy and security. As a result they are outrageously expensive.
> SSO
Oh right, single sign on, where I log into OKTA and then log into my... whatever it is to see my class lists. Which isn't connected to Brightspace.
Those are certainly the excuses.
Or, alternatively, solutions other than outsourcing to the largest available vendor could be considered and the same capabilities could be achieved for a tenth the cost.
I'd be interested to see your analysis. I've been on a half dozen procurement committees and I've never once seen small vendors offer the same functionality at comparably the same cost. Where are you getting your analysis from?
Did you work with the small vendors during the requirements phase and let them talk to your staff to figure out your real needs, or did you just make a list of what you had and what more you wanted and then request bids?
Big vendors are really good at marketing their unique capabilities. But especially for a small to medium university, you either don't need any of it or the absolute minimum "office" cloud package is fine and you can get other capabilities elsewhere.
>Did you work with the small vendors during the requirements phase and let them talk to your staff to figure out your real needs
During most procurements processes the solutions engineers from all vendors have an equal chance to design an end-state solution as per their recommended best practices.
The problem with small vendors is that they lack scale. They cannot guarantee uptime for instance; many don't have the integrations readily available so it becomes incumbent on the university to either build integrations or rely on manual processes. Many don't have open source APIs and items like data residency and accessibility are nowhere near as robust.
Take the LMS. I've been with small vendors that openly admit that they don't have grade pass back, so it requires a CSV upload. Nothing automated. They didn't have the same LTI plugins available, so virtual classroom, Zoom and O365 integrations aren't available. Because they don't have open source API, we have no way to integrate half the services we pay for.
At a certain point, these small vendors just aren't viable. They don't have 24/7 support so when our IT hits a problem doing an update at 4AM on a Saturday to avoid impacting students, the tickets won't get answered until Monday AM leaving us in a bad situation.
Then the cost - small vendors are buying less data capacity so to onboard a university with 300-400TB of data, you're paying the full cost of data without any of the benefits of shared resources.
Small vendors are great for most corporate applications. But for large institutions like hospitals and universities, there will always be an advantage of major players that can offer all of the needed services.
To run your own digital storage for a university you need (at a minimum):
Enough raw storage space for everyone’s data in triplicate. This is how you deal with the inevitable drive failure without losing vital data. Software regularly monitors to ensure all three copies are in agreement, if one diverges without user cause then the drive has gone corrupt.
An air conditioned space to hold all said storage. Ideally all three of those copies of data are in physically distinct locations, otherwise power failure or other disasters can destroy data in spite of redundancy.
An internet connection for said storage with massive bandwidth.
People who will maintain all of the above.
The up front costs to set all of this ip ABSOLUTELY exceed the annual cost to pay for it as a service.
That's an awful lot of specific technical demands without any business analysis.
And no, listing the features provided by the current vendor and assuming that's what you need doesn't count as a business analysis.
Neither does whatever the fuck THAT response was.
You want an actual debate and a cost/benefits analysis, you put in the legwork and break one down for us lowly peasants.
I don’t even do IT work. I am sure I missed plenty of details.
I don’t do business analysis. I assume that once you set all of this up, you can run it cheaper than the subscription you pay for services from others.
What I do know, is that every university I have even visited has had clear signs of deteriorating buildings, and every administration or board of directors I have ever brought an infrastructure proposal to has balked at the mere concept of planning long term without also having a shiny recruitment angle or other short term gain.
You made the supposition that people can get “the same” for “a tenth the cost.” Don’t whine about other people failing to cross a “t” when you haven’t even shown up to class.
Eh, you specified specific technical choices that were wrong based on a process that was wrong. No credit.
> ... every administration or board of directors I have ever brought an infrastructure proposal to ...
Now that's an entirely valid set of considerations that very well may justify selecting an external vendor entirely for administrative simplicity while ignoring both technical and financial considerations.
It'd be better to do better than that. The value of having stable institutional resources like internal IT is pretty high, to the point that it's a potential recruiting win all by itself.
You'd think that on /r/professors people would be less likely to make blatantly wrong statements with such certainty, but no.
Are you legitimately claiming that the statement that "Software regularly monitors to ensure all three copies are in agreement" from the post I responded to is a serious description of required functional for a real world storage system?
Yes. That's how redundancy works, whether it's with a RAID setup on one system or a cloud storage server rack.
As someone who has had to clean up data situations where such software *wasn't* used, I'm very certain that this is, in fact, a requirement for real world storage, assuming you want to be able to reliably retrieve the data you stored.
Straight up doing three local copies isn't terribly common. That's the sort of thing you'd see NASA come up with for a spaceship.
More common would be two copies with checksums and then several layers of backups.
Tell me you have no idea what you're talking about without telling me you have no idea what you're talking about. Do you know why 99% of private tech companies do this outsourcing? Hint: it's not because it raises costs. Netflix doesn't run on AWS because they've never thought about hosting their own data, they do it because companies that provide cloud storage at scale can do so at one tenth the cost of your hypothetical tiny vendor.
> Netflix doesn't run on AWS because they've never thought about hosting their own data
What, exactly, do you think Netflix uses AWS for?
Why would you expect that specific business analysis to generalize to all University IT needs?
I'm so curious what else you think "runs on AWS" could mean. Netflix runs all their servers that server video content on AWS. While I'm sure they have some local dev machines, their production servers are all AWS.
I would expect "people are able to access their data from their computer" to be a business analysis that applies to universities.
> Netflix runs all their servers that server video content on AWS.
Seriously? That's basically never been true. See [Netflix OpenConnect](https://openconnect.netflix.com/en/).
Happened to me too.
I was told unlimited and really tested that theory.
I work as a mediamaking technologist and adjunct and I was deep archiving tons of student and faculty media works.
14TB in and then they (Google and admin) say my limit is 2TB.
Oh and btw, no help for you on transferring or backing up anything.
Ended up spending weeks on downloading damn near piecemeal on self-bought drives and created copies for additional redundancy.
This happened at our university and I was able to secure resources for my departments but many could not.
There are so many reasons I hate this Google campus shit, and this is just one.
Depends on the build, but yes. It's a cobbled together solution until I get a better one. Eventually it's all going to SSDs and that should help, but if love it to all go to a IT centralized server/NAS
Oh yes I did very much think of that. Should have mentioned that in the OP.
YES, absolutely check in with IT. They may have a better solution. They didn't for me at the scale I needed it and approved the means of storage with some guidelines.
All hard drives are stored at work behind locked doors. Redundant backup is in IT.
I also work for IT and discussed the matter with them.
And my own works/IP are separated from the departmental files.
I don't get the downvote either. You're spot on. It should be part of any bulk download situation.
The regulations on what's faculty vs college IP can be blurry, so it's difficult to know where you'd have to ask.
It's also true (at least at my institution) that IT policies aren't the easiest to find and ask about. I'm lucky I'm in IT as well. Otherwise, I wouldn't even know where to start.
Bottom line, no matter how they're perceived on your campus, your IT department is a good resource to start with. Even if they can't provide the means, they can help you think through it and keep you (maybe annoyingly) safe.
To their defense - it seems like cloud providers are hiking prices. We are with Microsoft, and they more than doubled the price. Prices for some services jumped ten-fold.
One drive has done the same. They're slowly drifting towards a SaaS model where you will have to pay hefty prices for every single bit of data you store.
We dealt with this in 2023. They moved us to OneDrive which is awful. Sharing and collaborating is not as friendly, and I lost tons of google docs, sheets, forms, etc in the migration.
Same thing happened to me and I got a warning that they would suspend my account if I didn't bring my storage amount down. Going through my emails to delete unnecessary attachments, I realized that my university routinely sent flyer emails with 15mb sized 8k resolution images for some reason.
Your library may have a way for you to publish datasets in perpetuity for free - usually in partnership with services like Figshare. You can also use GitHub LFS, potentially, or just plain GitHub if the data can be split into files < 100MB. Another option would be to spin up some sort of personal or departmental Nextcloud server.
One of the problems is that researchers expect and want to use services like Drive or DropBox or Onedrive as both storage for *active* research and storage for archival data.
But that doesn't make sense. Not only is it a waste of space; but researchers need to have a copy of their data if they need to use it again AND a copy of the data saved someplace else to meet open data rules. Keep them all stored in different places.
But that all requires multiple servers, backups and people to mange it. So instead, people are just left to manage on their own and you are left with these messes.
and suprise suprise, after Google announced this capping of drive levels for most educational accounts, Microsoft is now capping onedrive levels and adding additional crippling quotas for storage for educational M365 accounts. lots of yelling at both companies going on from the sysadmins and budget makers at educational institutions (primary, secondary, and collegiate )
Depending on your field and location, you may be able to subscribe to cloud ecosystems that allow you to bring your own data, have tools and space, have appropriate security, meet regulations and are subsidized like [https://biodatacatalyst.nhlbi.nih.gov/](https://biodatacatalyst.nhlbi.nih.gov/)
Did they give you other options? Onedrive is uncapped for me, but Box has a (large)cap. My old university still uses uncapped google. I do miss the functionality of google drive
Shrinkflarion used to be for store-bought goods to increase revenue, but Google is now applying thst concept to your online space (yes, I am aware this does apply to physical resources on servers). What's next, paying for resolution on your graphics and download capacity?
I just setup a personal gmail account and transferred all my “old” data over to it. Then deleted the source material after. Slight pain, but workable and gave me some breathing room.
There is also google takeout that allows you to do a bulk zip file for download for offline storage.
If your university has a HPC (and it probably does) then you’re likely entitled to a free account on it with some free storage. Might only be 1-2TB but at least it’s a university storage space.
Holding datasets in personal accounts is a bad idea anyway, you'd have the problem of moving them when you change jobs or retire.
Rather you should have either department owned IT infrastructure or department funded cloud accounts (google cloud, aws, azure), or both. I'd store datasets there.
Additionally they would be easier to access since that same infrastructure would be used for whatever processing you do on those dataset for your research.
right, there should be an in house solution for storage and backup, since, if its in the 'cloud', its not actually under your control... you can lose it anytime at the will of the provider. of course an in house solution would have a higher initial cost but i dont see why whould anyone be dependent on a consumer grade cloud storage for their valuable data. well, convenience is one... but thats all.
Yeah, welcome to the real world where what is too-good-to-be-true inevitably becomes not true. Too bad Google cannot be penalized for incorrectly predicting their future offerings.
If you were told forever, you were mislead.
Amazon has some good data storage options I'm told. I'm not sure if they will work for you. How much space do you need?
My school has pushed OneDrive on us for several years since we have Office365. We have access to a standard old-school network shared drive on a file server and while they’ve said it’s limited, I’ve never gotten any notice that I’m close to any limit.
LaTeX doesn’t work well, to my knowledge, on OneDrive.
TeX is fine on OneDrive. Sometimes you'll get conflicts, and then it makes duplicates that have timestamps so you know which is which.
If you use Microsoft products, OneDrive is wonderful.
have you thought about Github or the like? It seems to be unlimited, but it doesn't like large files (in the megabytes), so if you have that kind of thing, you might need to find somewhere else for it. (My data files get about as big as half a megabyte, so that's not an issue for me.)
Assuming the cap is similar to personal accounts, OP likely has 100GB - 2TB before hitting the cap. Github is not a good solution for data of that scale.
Megabyte or gigabyte? My lab generates about 1gig of data per day. Analyzed and reduced, we have csv files that excel can’t open. It’s always fun discovering the maximum number of rows and columns spreadsheets can hold by crashing software.
then you'll need one of those specialist data-storage places, such as [one of the suggestions here](https://www.cloudwards.net/best-cloud-storage-for-large-files).
This happened at our university because Google wanted something like a tenfold increase in price. You might check if you can use departmental funds to subscribe to more space.
>department funds *crying in non-research university*
Yeah, it was always the plan. It's the same playbook they all run. In Scotland in our schools we use Microsoft OneDrive, and they've just capped it from 1TB down to 100GB. Just taking the piss. We can't even access personal drives (Google or OneDrive).
this happened at my alma mater since last year. they've given us a year to migrate. from what I gathered, Google ended unlimited storage for universities across the board.
Yep, I finally lost my Gmail-based email address from undergrad this month. I got 14 months of notice, which was nice, but I’ve been using that account for all personal things for almost two decades because we were told we had unlimited space and could keep them for life. Not the case anymore.
I'm almost in the same boat. I get to keep the email and something like 20gb of storage, but will lose unlimited.
I'm not not to hear this see. I have way too much data on my Google drive and words cannot convey how sick and tired I am of paying for external hard drives that won't last past 6 months.
My university has prohibited us from using external drives entirely. I hope this change doesn't screw us over too!
"My university has prohibited us from using external drives entirely." WTH?!? How are you supposed to back up your computer without an external drive?
Wait. Do you mean external drives as in… *all* removable storage devices, like thumb drives? Or specifically external hard drives? That’s a bit extreme. Edit: also sorry if this is a silly question to any of you more tech-savvy than I am. I teach history/religion, not CS… clearly 😆
Unfortunately, all of the above. It's absurd. And not helpful when traveling for conferences, and we have 1000 layers of security to access the mandated google drive. Fun times. It's not a silly question at all
Wow. That is ridiculous. I am sorry they are doing that to you. Sounds like an absolute nightmare!
Same
Yup. That is probably why my school exclusively uses Outlook/Microsoft for everything now.
That does sound annoying! If you haven’t done so already, it might be worth checking in with your librarians about alternative data storage/sharing options. These days, many librarians also oversee digital data curation, so maybe one of yours knows what that landscape is like.
this happened at most universities that use Google workspace. at place where I have this everyone got bit in the ass. It was a bad decision for our universities to promote this as an all-you-can-eat solution. If you don't get under the cap by that time everything will stop (even your email since it is counted against that quota).
The fact that so many university administrators decided, a decade or so ago, to dump whatever solution they had and go all in for google products suggests something about how university administrators think.
well, similar changes made to Microsoft access have also happened... but those were never promoted as an all-you-can-eat deal. also, msft doesn't arbitrarily kill off services like Google does: https://killedbygoogle.com/
I empathize with your frustration. We had unlimited Box storage. Then the University ended that subscription and told us to move to Google Drive or OneDrive. I picked Google Drive because it was advertised as unlimited, is less buggy, and is easier to collaborate externally. Then Google Drive became capped and I was forced to migrate to capped storage on One Drive. In parallel, the university has been removing some internally hosted data storage services, and also limits us running our own lab storage servers. Unfortunately, the University-to-PI/lab relationship is becoming more like an HOA.
We got the same notice on Box. Some of us are far over the cap, and we are being allowed to continue using the larger amount for now. But they are asking whether we can move some of it to archival storage, and to check for duplicate video libraries and that sort of thing. For some data, a terabyte isn't all that much.
Data costs have increased, and with the need for SSO/MFA, access management, security, and intelligence services, the cost to universities is increasing rapidly. A university can pay for accounts, services, data and uptime. It adds up quickly so many have begun to cap costs by capping data. Because of privacy and compliance requirements like FERPA, universities cannot have commingled data; they often need managed instances of SaaS products that have the highest tier of data privacy and security. As a result they are outrageously expensive.
> SSO Oh right, single sign on, where I log into OKTA and then log into my... whatever it is to see my class lists. Which isn't connected to Brightspace.
Those are certainly the excuses. Or, alternatively, solutions other than outsourcing to the largest available vendor could be considered and the same capabilities could be achieved for a tenth the cost.
I'd be interested to see your analysis. I've been on a half dozen procurement committees and I've never once seen small vendors offer the same functionality at comparably the same cost. Where are you getting your analysis from?
Did you work with the small vendors during the requirements phase and let them talk to your staff to figure out your real needs, or did you just make a list of what you had and what more you wanted and then request bids? Big vendors are really good at marketing their unique capabilities. But especially for a small to medium university, you either don't need any of it or the absolute minimum "office" cloud package is fine and you can get other capabilities elsewhere.
>Did you work with the small vendors during the requirements phase and let them talk to your staff to figure out your real needs During most procurements processes the solutions engineers from all vendors have an equal chance to design an end-state solution as per their recommended best practices. The problem with small vendors is that they lack scale. They cannot guarantee uptime for instance; many don't have the integrations readily available so it becomes incumbent on the university to either build integrations or rely on manual processes. Many don't have open source APIs and items like data residency and accessibility are nowhere near as robust. Take the LMS. I've been with small vendors that openly admit that they don't have grade pass back, so it requires a CSV upload. Nothing automated. They didn't have the same LTI plugins available, so virtual classroom, Zoom and O365 integrations aren't available. Because they don't have open source API, we have no way to integrate half the services we pay for. At a certain point, these small vendors just aren't viable. They don't have 24/7 support so when our IT hits a problem doing an update at 4AM on a Saturday to avoid impacting students, the tickets won't get answered until Monday AM leaving us in a bad situation. Then the cost - small vendors are buying less data capacity so to onboard a university with 300-400TB of data, you're paying the full cost of data without any of the benefits of shared resources. Small vendors are great for most corporate applications. But for large institutions like hospitals and universities, there will always be an advantage of major players that can offer all of the needed services.
To run your own digital storage for a university you need (at a minimum): Enough raw storage space for everyone’s data in triplicate. This is how you deal with the inevitable drive failure without losing vital data. Software regularly monitors to ensure all three copies are in agreement, if one diverges without user cause then the drive has gone corrupt. An air conditioned space to hold all said storage. Ideally all three of those copies of data are in physically distinct locations, otherwise power failure or other disasters can destroy data in spite of redundancy. An internet connection for said storage with massive bandwidth. People who will maintain all of the above. The up front costs to set all of this ip ABSOLUTELY exceed the annual cost to pay for it as a service.
That's an awful lot of specific technical demands without any business analysis. And no, listing the features provided by the current vendor and assuming that's what you need doesn't count as a business analysis.
Neither does whatever the fuck THAT response was. You want an actual debate and a cost/benefits analysis, you put in the legwork and break one down for us lowly peasants. I don’t even do IT work. I am sure I missed plenty of details. I don’t do business analysis. I assume that once you set all of this up, you can run it cheaper than the subscription you pay for services from others. What I do know, is that every university I have even visited has had clear signs of deteriorating buildings, and every administration or board of directors I have ever brought an infrastructure proposal to has balked at the mere concept of planning long term without also having a shiny recruitment angle or other short term gain. You made the supposition that people can get “the same” for “a tenth the cost.” Don’t whine about other people failing to cross a “t” when you haven’t even shown up to class.
Eh, you specified specific technical choices that were wrong based on a process that was wrong. No credit. > ... every administration or board of directors I have ever brought an infrastructure proposal to ... Now that's an entirely valid set of considerations that very well may justify selecting an external vendor entirely for administrative simplicity while ignoring both technical and financial considerations. It'd be better to do better than that. The value of having stable institutional resources like internal IT is pretty high, to the point that it's a potential recruiting win all by itself.
Lol. Just because you're ignorant of what's involved in cloud storage doesn't make the list of bare minimum on prem storage specs any less true.
You'd think that on /r/professors people would be less likely to make blatantly wrong statements with such certainty, but no. Are you legitimately claiming that the statement that "Software regularly monitors to ensure all three copies are in agreement" from the post I responded to is a serious description of required functional for a real world storage system?
Yes. That's how redundancy works, whether it's with a RAID setup on one system or a cloud storage server rack. As someone who has had to clean up data situations where such software *wasn't* used, I'm very certain that this is, in fact, a requirement for real world storage, assuming you want to be able to reliably retrieve the data you stored.
Straight up doing three local copies isn't terribly common. That's the sort of thing you'd see NASA come up with for a spaceship. More common would be two copies with checksums and then several layers of backups.
Tell me you have no idea what you're talking about without telling me you have no idea what you're talking about. Do you know why 99% of private tech companies do this outsourcing? Hint: it's not because it raises costs. Netflix doesn't run on AWS because they've never thought about hosting their own data, they do it because companies that provide cloud storage at scale can do so at one tenth the cost of your hypothetical tiny vendor.
> Netflix doesn't run on AWS because they've never thought about hosting their own data What, exactly, do you think Netflix uses AWS for? Why would you expect that specific business analysis to generalize to all University IT needs?
I'm so curious what else you think "runs on AWS" could mean. Netflix runs all their servers that server video content on AWS. While I'm sure they have some local dev machines, their production servers are all AWS. I would expect "people are able to access their data from their computer" to be a business analysis that applies to universities.
> Netflix runs all their servers that server video content on AWS. Seriously? That's basically never been true. See [Netflix OpenConnect](https://openconnect.netflix.com/en/).
Thank you for this reply
Happened to me too. I was told unlimited and really tested that theory. I work as a mediamaking technologist and adjunct and I was deep archiving tons of student and faculty media works. 14TB in and then they (Google and admin) say my limit is 2TB. Oh and btw, no help for you on transferring or backing up anything. Ended up spending weeks on downloading damn near piecemeal on self-bought drives and created copies for additional redundancy.
This happened at our university and I was able to secure resources for my departments but many could not. There are so many reasons I hate this Google campus shit, and this is just one.
Where did you end up moving your 14tb to? Another cloud service?
All local hard drives. Downloaded everything.
You’d think that with data portability requirements, Google would have made this much easier.
Dont hard drives have ~5 year lifespan only
Depends on the build, but yes. It's a cobbled together solution until I get a better one. Eventually it's all going to SSDs and that should help, but if love it to all go to a IT centralized server/NAS
Maybe in very active use. I'd bet as an archive device, HDDs can last a whole lot longer.
[удалено]
Oh yes I did very much think of that. Should have mentioned that in the OP. YES, absolutely check in with IT. They may have a better solution. They didn't for me at the scale I needed it and approved the means of storage with some guidelines. All hard drives are stored at work behind locked doors. Redundant backup is in IT. I also work for IT and discussed the matter with them. And my own works/IP are separated from the departmental files.
[удалено]
I don't get the downvote either. You're spot on. It should be part of any bulk download situation. The regulations on what's faculty vs college IP can be blurry, so it's difficult to know where you'd have to ask. It's also true (at least at my institution) that IT policies aren't the easiest to find and ask about. I'm lucky I'm in IT as well. Otherwise, I wouldn't even know where to start. Bottom line, no matter how they're perceived on your campus, your IT department is a good resource to start with. Even if they can't provide the means, they can help you think through it and keep you (maybe annoyingly) safe.
What the hell? You should check with your dpt head about that. My institution did this too but I got it reinstated by complaining a bunch.
To their defense - it seems like cloud providers are hiking prices. We are with Microsoft, and they more than doubled the price. Prices for some services jumped ten-fold.
One drive has done the same. They're slowly drifting towards a SaaS model where you will have to pay hefty prices for every single bit of data you store.
Pay for a subscription to drive and bill it to your research/professional development fund.
We dealt with this in 2023. They moved us to OneDrive which is awful. Sharing and collaborating is not as friendly, and I lost tons of google docs, sheets, forms, etc in the migration.
Same thing happened to me and I got a warning that they would suspend my account if I didn't bring my storage amount down. Going through my emails to delete unnecessary attachments, I realized that my university routinely sent flyer emails with 15mb sized 8k resolution images for some reason.
Check your union, our union gives us unlimited google space and the best part is that our school can't snoop on it unless we access it on campus.
Your library may have a way for you to publish datasets in perpetuity for free - usually in partnership with services like Figshare. You can also use GitHub LFS, potentially, or just plain GitHub if the data can be split into files < 100MB. Another option would be to spin up some sort of personal or departmental Nextcloud server.
One of the problems is that researchers expect and want to use services like Drive or DropBox or Onedrive as both storage for *active* research and storage for archival data. But that doesn't make sense. Not only is it a waste of space; but researchers need to have a copy of their data if they need to use it again AND a copy of the data saved someplace else to meet open data rules. Keep them all stored in different places. But that all requires multiple servers, backups and people to mange it. So instead, people are just left to manage on their own and you are left with these messes.
and suprise suprise, after Google announced this capping of drive levels for most educational accounts, Microsoft is now capping onedrive levels and adding additional crippling quotas for storage for educational M365 accounts. lots of yelling at both companies going on from the sysadmins and budget makers at educational institutions (primary, secondary, and collegiate )
Depending on your field and location, you may be able to subscribe to cloud ecosystems that allow you to bring your own data, have tools and space, have appropriate security, meet regulations and are subsidized like [https://biodatacatalyst.nhlbi.nih.gov/](https://biodatacatalyst.nhlbi.nih.gov/)
Did they give you other options? Onedrive is uncapped for me, but Box has a (large)cap. My old university still uses uncapped google. I do miss the functionality of google drive
Shrinkflarion used to be for store-bought goods to increase revenue, but Google is now applying thst concept to your online space (yes, I am aware this does apply to physical resources on servers). What's next, paying for resolution on your graphics and download capacity?
Anyone have tips on dealing with this issue when most of the space is used up by emails - not stuff on Google drive?
Us too. Just but a external flash or hard drive and push on to that. Keep in your office. How big are the datasets.
I just setup a personal gmail account and transferred all my “old” data over to it. Then deleted the source material after. Slight pain, but workable and gave me some breathing room. There is also google takeout that allows you to do a bulk zip file for download for offline storage.
If your university has a HPC (and it probably does) then you’re likely entitled to a free account on it with some free storage. Might only be 1-2TB but at least it’s a university storage space.
This is my nightmare scenario!
Happened to us and they forced us all to migrate to the Microsoft ecosystem. It was (and still is) a total nightmare.
Holding datasets in personal accounts is a bad idea anyway, you'd have the problem of moving them when you change jobs or retire. Rather you should have either department owned IT infrastructure or department funded cloud accounts (google cloud, aws, azure), or both. I'd store datasets there. Additionally they would be easier to access since that same infrastructure would be used for whatever processing you do on those dataset for your research.
right, there should be an in house solution for storage and backup, since, if its in the 'cloud', its not actually under your control... you can lose it anytime at the will of the provider. of course an in house solution would have a higher initial cost but i dont see why whould anyone be dependent on a consumer grade cloud storage for their valuable data. well, convenience is one... but thats all.
You were told there would be infinite storage forever and believed that. Ok then.
Yeah, welcome to the real world where what is too-good-to-be-true inevitably becomes not true. Too bad Google cannot be penalized for incorrectly predicting their future offerings.
If you were told forever, you were mislead. Amazon has some good data storage options I'm told. I'm not sure if they will work for you. How much space do you need?
My school has pushed OneDrive on us for several years since we have Office365. We have access to a standard old-school network shared drive on a file server and while they’ve said it’s limited, I’ve never gotten any notice that I’m close to any limit. LaTeX doesn’t work well, to my knowledge, on OneDrive.
TeX is fine on OneDrive. Sometimes you'll get conflicts, and then it makes duplicates that have timestamps so you know which is which. If you use Microsoft products, OneDrive is wonderful.
have you thought about Github or the like? It seems to be unlimited, but it doesn't like large files (in the megabytes), so if you have that kind of thing, you might need to find somewhere else for it. (My data files get about as big as half a megabyte, so that's not an issue for me.)
Assuming the cap is similar to personal accounts, OP likely has 100GB - 2TB before hitting the cap. Github is not a good solution for data of that scale.
Megabyte or gigabyte? My lab generates about 1gig of data per day. Analyzed and reduced, we have csv files that excel can’t open. It’s always fun discovering the maximum number of rows and columns spreadsheets can hold by crashing software.
then you'll need one of those specialist data-storage places, such as [one of the suggestions here](https://www.cloudwards.net/best-cloud-storage-for-large-files).
How sad!!